Patch GPG to support SOCKS proxies
Currently, GPG supports using HTTP proxies, but not using SOCKS proxies. Before we get rid of Polipo, we should add support for SOCKS proxies to GPG so Windows users have some hope of torifying GPG. (Users of most Unixoid systems, now including MacOS and FreeBSD, can use torsocks to torify GPG.)
Activity
Replying to mikeperry:
Is GPG actually in our bundles?
No, but some users torify GPG using its HTTP proxy support, and Windows users who need to torify GPG can't use torsocks for that purpose.
We have a libtorsocks that ioerror adapted from torsocks. We maybe able to use some of its functionality in a patch to GPG.
The licenses are compatible.
That's great. The reality is this shouldn't block us from making releases without Polipo though. There are probably 2 people who use the gpg command line on Windows with Tor. It is almost certain that the other 7 GPG users on Windows use a GUI and would never even know to set proxy settings of any sort for the execution of gpg itself.
Trac:
Cc: N/A to tor@flamsmark.com-
At best this is a Torify shortcoming.. Certainly not a Tor bundles issue.
Trac:
Parent: #2844 (closed) to N/A
Component: Tor bundles/installation to Torify
Resolution: N/A to not a bug
Status: new to closed As pointed out by David Shaw you can use socks proxies with gpg on non-Windows systems if your curl version is >= 7.21.7
http://lists.gnupg.org/pipermail/gnupg-devel/2012-September/026923.html
See also #6940 (closed)
Bad news - if the GPG client has no SOCKS5 support and you try to use the SOCKS5 support it will fail badly.
It has DNS leaks:
DNS Standard query SRV _pgpkey-http._tcp.pool.sks-keyservers.net DNS Standard query response, No such nameSo in a sense we have gpg support for SOCKS proxies but versions that don't have it won't fail before harming users.
I don't think this is a solved problem on Windows.
On Mac OS X 10.5.8, I see that the popular gpg package ( https://www.gpgtools.org ) links against libcurl/7.16.3 and so it does not work. I suspect that other versions of Mac OS X will have other versions of libcurl. A version table would be nice - does anyone have one offhand?
Cleanest imaginable solution: #6060 (moved) add http proxy support to Tor
Can you do it? Can perhaps portions of privoxy or poliop source code be used?
Trac:
Cc: tor@flamsmark.com to tor@flamsmark.com, adrelanos@riseup.netReplying to ioerror:
Bad news - if the GPG client has no SOCKS5 support and you try to use the SOCKS5 support it will fail badly.
This means we can not statically configure gpg to use socks5 proxies (in torbirdy).
Can we build an autodetection for socks support and change enigmail's config acordingly?
- enable socks5 proxy in enigmail's gpg options iff we have socks support
- do not set the socks proxy if we have no support for it (set a non-existing http proxy to fail secure)
I've heard that curl Version 7.21.4 is the version included in Mac OS X 10.8 - so that would likely be safe if it is properly used by gpgtools.
It appears that the versions of curl are as follows: Mac OS X 10.8.x - 7.24.0 Mac OS X 10.7.4 - 7.21.4 Mac OS X 10.6 - 7.19.0 Mac OS X 10.5 - 7.16.2 Mac OS X 10.4 (intel) - 7.13.0
I'm guessing that information based on extracting version numbers from Apple's developer man page website. Painful and likely error prone.
If that is correct, I guess no version of gpg on Mac OS X 10.4.x -> 10.7.x would be likely to support such a proxy. It may be the case that Mac OS X 10.8.x has a newer curl release but I'm not sure.
Replying to tagnaq:
Replying to ioerror:
Bad news - if the GPG client has no SOCKS5 support and you try to use the SOCKS5 support it will fail badly.
This means we can not statically configure gpg to use socks5 proxies (in torbirdy).
I'm not sure of that but I understand the caution. I think that we just need to detect this (common) case and do something smart.
Can we build an autodetection for socks support and change enigmail's config acordingly?
- enable socks5 proxy in enigmail's gpg options iff we have socks support
Sounds like a bug for enigmail - want to open it? :)
- do not set the socks proxy if we have no support for it (set a non-existing http proxy to fail secure)
That seems OK, I guess.
Well - things just appear to get worse and worse here - I see the same SRV DNS leaks (DNS Standard query SRV _pgpkey-http._tcp.pool.sks-keyservers.net) even when we use an HTTP proxy. See this bug for how I setup a local HTTP proxy with shim: https://trac.torproject.org/projects/tor/ticket/6060#comment:8
It seems like there isn't much of a patch required - either the current version of GPG is built against a curl with socks:// support or it isn't; either way, we'll need to smoke out all the stray dnsleaks.
I presume pka-lookups, cert, ldap modes of keylookup will likely also leak DNS.
In the case of pka-lookups it looks like internally (g10/gpg.c) it may require that we set 'no-auto-key-retrieve' - I'm not sure of the best way to trigger such a lookup. If anyone has suggestions, I'd love to know how to trigger it.
So now we're up to two DNS leak plugging key-server options:
no-auto-key-retrieve,no-try-dns-srvI'll next look into the 'cert, ldap' code to see how it leaks and report back.
I tried to make it leak with the following ldap request:
gpg --keyserver ldap://keyserver.pgp.com --keyserver-options no-auto-key-retrieve,no-try-dns-srv,http-proxy=http://127.0.0.1:8119,debug,verbose --search jacob@appelbaum.netIt leaks DNS:
DNS Standard query AAAA keyserver.pgp.com DNS Standard query AAAA keyserver.pgp.com.localdomain DNS Standard query A keyserver.pgp.comI also tried with SOCKS:
gpg --keyserver ldap://keyserver.pgp.com --keyserver-options no-auto-key-retrieve,no-try-dns-srv,http-proxy=socks5-hostname://127.0.0.1:9050,debug,verbose --search jacob@appelbaum.netThat also appears to break out of the proxy entirely. Epic.
I guess I might change my build suggestion above to something more restrictive:
./configure \ --disable-dns-cert \ --disable-dns-pka \ --disable-dns-srv \ --disable-ldapOn to some good news, I guess.
The following examples don't leak DNS and properly use the HTTP proxy.
x-hkp://
gpg --keyserver x-hkp://pool.sks-keyservers.net --keyserver-options no-auto-key-retrieve,no-try-dns-srv,http-proxy=http://127.0.0.1:8119,debug,verbose --recv-key 0x4193A197x-broken-hkp://
gpg --keyserver x-broken-hkp://kpool.sks-keyservers.net --keyserver-options no-auto-key-retrieve,no-try-dns-srv,http-proxy=http://127.0.0.1:8119,debug,verbose --recv-key 0x4193A197The 'broken-http-proxy' key server option:
pg --keyserver hkp://kpool.sks-keyservers.net --keyserver-options broken-http-proxy,no-auto-key-retrieve,no-try-dns-srv,http-proxy=http://127.0.0.1:8119,debug,verbose --recv-key 0x4193A197An FTP key server:
gpg --keyserver ftp://keyserver.pgp.com --keyserver-options broken-http-proxy,no-auto-key-retrieve,no-try-dns-srv,http-proxy=http://127.0.0.1:8119,debug,verbose --recv-key 0x4193A197A bullshit protocol name I just futzed up:
gpg --keyserver ///://keyserver.pgp.com --keyserver-options broken-http-proxy,no-auto-key-retrieve,no-try-dns-srv,http-proxy=http://127.0.0.1:8119,debug,verbose --recv-key 0x4193A197 A special local host name (see line 359 of g10/keyserver.c): {{{ gpg --keyserver x-hkp///keyserver.pgp.com --keyserver-options broken-http-proxy,no-auto-key-retrieve,no-try-dns-srv,http-proxy=http://127.0.0.1:8119,debug,verbose --recv-key 0x4193A197That last one is funny and causes gpg to do something odd (looks like a bug to me...):
> GET http://x-hkp:11371///keyserver.pgp.com/pks/lookup?op=get&options=mr&search=0x4193A197 HTTP/1.1Lucky for us - the proxy support is respected in all of the above cases.
Err - bad formatting - I meant to do it correctly.
A bullshit protocol name I just futzed up:
gpg --keyserver ///://keyserver.pgp.com --keyserver-options broken-http-proxy,no-auto-key-retrieve,no-try-dns-srv,http-proxy=http://127.0.0.1:8119,debug,verbose --recv-key 0x4193A197A special local host name (see line 359 of g10/keyserver.c):
gpg --keyserver x-hkp///keyserver.pgp.com --keyserver-options broken-http-proxy,no-auto-key-retrieve,no-try-dns-srv,http-proxy=http://127.0.0.1:8119,debug,verbose --recv-key 0x4193A197That last one is funny and causes gpg to do something odd (looks like a bug to me...):
> GET http://x-hkp:11371///keyserver.pgp.com/pks/lookup?op=get&options=mr&search=0x4193A197 HTTP/1.1Lucky for us - the proxy support is respected in all of the above cases.
To be specific - I need the above tests run (with tcpdump logging traffic) for gpg built against libcurl >= 7.21.7 - this should help us to see if the SOCKS5 proxy support is working properly.
Here are the commands to run:
x-hkp://
gpg --keyserver x-hkp://pool.sks-keyservers.net --keyserver-options no-auto-key-retrieve,no-try-dns-srv,http-proxy=socks5-hostname://127.0.0.1:9050,debug,verbose --recv-key 0x4193A197x-broken-hkp://
gpg --keyserver x-broken-hkp://kpool.sks-keyservers.net --keyserver-options no-auto-key-retrieve,no-try-dns-srv,http-proxy=socks5-hostname://127.0.0.1:9050,debug,verbose --recv-key 0x4193A197ftp://
gpg --keyserver ftp://keyserver.pgp.com --keyserver-options no-auto-key-retrieve,no-try-dns-srv,http-proxy=socks5-hostname://127.0.0.1:9050,debug,verbose --recv-key 0x4193A197///://
gpg --keyserver ///://keyserver.pgp.com --keyserver-options no-auto-key-retrieve,no-try-dns-srv,http-proxy=socks5-hostname://127.0.0.1:9050,debug,verbose --recv-key 0x4193A197x-hkp///
gpg --keyserver x-hkp///keyserver.pgp.com --keyserver-options no-auto-key-retrieve,no-try-dns-srv,http-proxy=socks5-hostname://127.0.0.1:9050,debug,verbose --recv-key 0x4193A197ldap:// (expected breakout)
gpg --keyserver ldap://keyserver.pgp.com --keyserver-options no-auto-key-retrieve,no-try-dns-srv,http-proxy=socks5-hostname://127.0.0.1:9050,debug,verbose --recv-key 0x4193A197ldaps:// (expected breakout)
gpg --keyserver ldaps://keyserver.pgp.com --keyserver-options no-auto-key-retrieve,no-try-dns-srv,http-proxy=socks5-hostname://127.0.0.1:9050,debug,verbose --recv-key 0x4193A197Only the last two should fail and everything else should go through the SOCKS(4a,5) proxy without DNS leaks.
All of the above tests leak DNS requests when tested with gpg 1.4.12 and curl 7.24.0
Trac:
Cc: tor@flamsmark.com, adrelanos@riseup.net to tor@flamsmark.com, adrelanos@riseup.net, admin@insecure-complexity.comReplying to ioerror:
To be specific - I need the above tests run (with tcpdump logging traffic) for gpg built against libcurl >= 7.21.7 - this should help us to see if the SOCKS5 proxy support is working properly.
$ gpg2 --version gpg (GnuPG) 2.0.19
libcurl 7.27.0.
x-hkp://
Doesn't leak.
x-broken-hkp://
Doesn't leak.
ftp://
Doesn't leak.
///://
Doesn't leak.
x-hkp///
Doesn't leak.
ldap:// (expected breakout)
Leaks.
ldaps:// (expected breakout)
Leaks.
Replying to sukhbir:
Replying to ioerror:
To be specific - I need the above tests run (with tcpdump logging traffic) for gpg built against libcurl >= 7.21.7 - this should help us to see if the SOCKS5 proxy support is working properly.
$ gpg2 --version gpg (GnuPG) 2.0.19
libcurl 7.27.0.
x-hkp://
Doesn't leak.
x-broken-hkp://
Doesn't leak.
ftp://
Doesn't leak.
///://
Doesn't leak.
x-hkp///
Doesn't leak.
ldap:// (expected breakout)
Leaks.
ldaps:// (expected breakout)
Leaks.
What platforms?
Replying to ioerror:
Well - things just appear to get worse and worse here - I see the same SRV DNS leaks (DNS Standard query SRV _pgpkey-http._tcp.pool.sks-keyservers.net) even when we use an HTTP proxy.
This SRV DNS leak is analyzed and explained in chapter 3.6.6 (workaround see figure 4) http://bit.ly/qDZm7C
Replying to tagnaq:
Replying to ioerror:
Well - things just appear to get worse and worse here - I see the same SRV DNS leaks (DNS Standard query SRV _pgpkey-http._tcp.pool.sks-keyservers.net) even when we use an HTTP proxy.
This SRV DNS leak is analyzed and explained in chapter 3.6.6 (workaround see figure 4) http://bit.ly/qDZm7C
Can you paste any relevant details here?
i'm using these : Windows XP (SP3), Wireshark (1.8.1), Tor (0.2.2.39), polipo, Gpg4win (2.1.1 34299 beta), Thunderbird (12.0.1), Enigmail (1.4.2), other socks proxy server, other http-proxy server, Unbound (3rd party local DNS-Resolver), other network traffic monitoring software, etc.
i have tried this command-line:
gpg2.exe --display-charset utf-8 --keyserver-options http-proxy=socks5-hostname://127.0.0.1:9050,debug,verbose,no-cert-check --debug-level expert --verbose --no-emit-version --no-comments --throw-keyids --keyserver hkp://hkps.pool.sks-keyservers.net:443 --recv-keys 0xC96B350A
there is no DNS leak.
keyserver was created 1 day earlier, and supports HKPS, TLS secured & encrypted key exchange, queries, etc.
also tried other keyservers.
the "gpg2keys_hkp.exe" does connect with "tor.exe" at 127.0.0.1:9050. and Kleopatra shows success on uploading a new key.
but i could not download keys, or search with keyids !
no new circuit appears in vidalia's Tor Network Map !
Trac:
Username: Bry8Star
Cc: tor@flamsmark.com, adrelanos@riseup.net, admin@insecure-complexity.com to tor@flamsmark.com, adrelanos@riseup.net, admin@insecure-complexity.com, bry8star@yahoo.com-
per suggestion of users, increased Tor logging levels, then found out gnupg kept on trying to use 127.0.0.1:9050 as http-proxy, not as SOCKS proxy.
so used polipo (http-proxy, 127.0.0.1:8118), instead.
i used polipo-1.0.4.1-forbidden-1-win32.exe,
config is here.
tried this (using "hkp" scheme):
gpg2.exe --display-charset utf-8 --keyserver-options http-proxy=
https://127.0.0.1:8118,debug,verbose --debug-level expert --verbose --no-emit-version --no-comments --throw-keyids --keyserverhkp://pool.sks-keyservers.net --recv-keys 0x4193A197WORKS. new circuit appears in Vidalia's "Tor Network Map". and no DNS leak happens. the "gpg2keys_hkp.exe" binary creates connection:
gnupg (tool "gpg2keys_hkp.exe") -> polipo (8118) -> tor (9050) -> tor-net -> pool.sks-keyservers.net (11371) or round-robin keyserver (11371).
was able to send and receive keyids.
connection toward any "hkps" keyserver, did not succeed.
neither directly, nor via tor.
(most likely) windows edition gnupg, does not support HKPS yet.
i did not see any "hkps" selectable scheme,
in the scheme list of Kleopatra.
8559 13:30:24.xxxxxxxxx 192.168.0.10 34388 IP-KYSRVR-3 domain DNS 108 Standard query 0xb417 SRV _pgpkey-https._tcp.keyserver.hostname
8618 13:30:27.xxxxxxxxx 192.168.0.10 22959 IP-KYSRVR-3 domain DNS 120 Standard query 0x9204 DLV _pgpkey-https._tcp.keyserver.hostname.dlv.isc.org
8622 13:30:27.xxxxxxxxx IP-KYSRVR-3 domain 192.168.1.4 22959 DNS 822 Standard query response 0x9204 No such name
when tested hkps via polipo, no DNS leaks.
when tested for receiving keys, via "https" scheme,
then "gpg2keys_curl.exe" starts up,
but fails to communicate with destination keyserver,
communication error. server cert was specified.
wireshark shows no dns query performed.
keyserver which claimed they support https,
even those did not work.
so, gnupg for windows also lacks support
for this feature as well.
when tested for receiving keys, via "http" scheme,
then "gpg2keys_curl.exe" starts up,
connects with keyserver's ip-address,
but "no valid OpenPGP data found." is shown, and,
"total number processed" remains 0.
no DNS leaks.
wireshark shows no dns query performed !
keyserver which claimed they support http,
even those did not work.
so, gnupg for windows, also lacks support
for this feature as well.
Trac:
Username: Bry8Star 
This ticket's title seems obsolete, but I cannot rename it as it's unclear to me what is the expected outcome of this ticket. Now that the bundles don't ship a HTTP proxy anymore, I see two possible goals:
- documenting best practices regarding how to torify GnuPG, that is kill most of the content on https://trac.torproject.org/projects/tor/wiki/doc/TorifyHOWTO/GnuPG and replace it with simpler info, thanks to the insight provided on this ticket.
- improving GnuPG torification in Torbirdy's gpg.conf, and future bundles that may include GnuPG; e.g. should no-try-dns-srv be added there?
It seems that the way to go is to first evaluate Torbirdy's gpg.conf wrt. the insight provided on this ticket, improve it if needed, and once happy, use it as a reference in the best practices doc. Perhaps we need two tickets for that, the latter being blocked by the former.
-
Documenting gnupg --keyserver-options no-try-dns-srv: it's given to the keyserver helper (gpgkeys_hkp), which polls the SRV record _pgpkey-http._tcp.KEYSERVERNAME (or _pgpkey-https._tcp.KEYSERVERNAME) to find out the real host and port. This is the code:
1.4: http://git.gnupg.org/cgi-bin/gitweb.cgi?p=gnupg.git;a=blob;f=keyserver/gpgkeys_hkp.c;hb=refs/heads/STABLE-BRANCH-1-4#l884 2.0: http://git.gnupg.org/cgi-bin/gitweb.cgi?p=gnupg.git;a=blob;f=keyserver/gpgkeys_hkp.c;hb=refs/heads/STABLE-BRANCH-2-0#l884
In 2.1 it's currently missing, but Werner Koch sais he will work on it soon.
Trac:
Username: ilf GnuPG 2.1.5 is broken and hangs with a socks proxy (and it can't be socksified for some reason). Using curl works: https://unix.stackexchange.com/questions/196393/how-to-import-the-official-tor-gpg-keys
Trac:
Reviewer: N/A to N/A
Severity: N/A to Normal
Sponsor: N/A to N/AThe GnuPG 2.1 branch uses
dirmngrfor key server communication. According to its documentation it supports theuse-toroption. To quote the documentationThis option switches Dirmngr and thus GnuPG into "Tor mode" to route all network access via Tor (an anonymity network). WARNING: As of now this still leaks the DNS queries; e.g. to lookup the hosts in a keyserver pool. Certain other features are disabled if this mode is active.
The DNS leaks are probably caused by the dependence on SRV records to make these pools work and Tor not supporting these types of resource records.
For key server pools people can visit the SKS keyservers pool page. This page also mentions a hidden service. Using the hidden service bypasses the dependence on SRV records so someone would expect no DNS leaks. I've tested this solution by adding
keyserver hkp://jirk5u4osbsr34t5.onion use-torto my
~/.gnupg/dirmngr.conffile. The subsequent packet capture showed no DNS leaks during execution ofgnupg --searchandgnupg --refresh-keys.mentioned in issue #6060 (moved)
mentioned in issue #6940 (closed)
mentioned in issue #6965 (closed)
mentioned in issue tpo/core/tor#6060 (closed)
