... | ... | @@ -2,21 +2,19 @@ |
|
|
|
|
|
Here is an exploration of [[PluggableTransports|pluggable transports]], how they look on the wire.
|
|
|
|
|
|
Pluggable transports disguise Tor traffic for the evading of network censorship. Some transports try to make the traffic look like another protocol, and others try to make it look random. Some transports are aimed at evading IP-based blocks rather than content-based blocks. If you want to see what running Tor with a pluggable transport is like, download a recent version of Tor Browser (version 3.6 or later), and say '''Yes''' to the question ''Does your Internet Service Provider (ISP) block or otherwise censor connections to the Tor Network?''
|
|
|
* [https://www.torproject.org/download/download-easy Tor Browser download page]
|
|
|
Pluggable transports disguise Tor traffic for the evading of network censorship. Some transports try to make the traffic look like another protocol, and others try to make it look random. Some transports are aimed at evading IP-based blocks rather than content-based blocks. If you want to see what running Tor with a pluggable transport is like, download a recent version of Tor Browser (version 3.6 or later), and say **Yes** to the question _Does your Internet Service Provider (ISP) block or otherwise censor connections to the Tor Network?_
|
|
|
* [Tor Browser download page](https://www.torproject.org/download/download-easy)
|
|
|
|
|
|
We'll look at hex dumps and text logs, outside and inside encryption, sometimes paying attention to packet boundaries and sometimes ignoring them, in order to discover what makes each transport unique and wonderful.
|
|
|
|
|
|
= Ordinary Tor =
|
|
|
# Ordinary Tor
|
|
|
|
|
|
{{{
|
|
|
#!comment
|
|
|
```
|
|
|
tor 0.2.4.21-1
|
|
|
}}}
|
|
|
```
|
|
|
|
|
|
Why use pluggable transports? It's because it's possible to fingerprint ordinary Tor traffic based on byte patterns that appear in it. Here's a hex dump of the first thing that a client sends to its entry node. It is actually a TLS [https://tools.ietf.org/html/rfc5246#section-7.4.1.2 Client Hello] message, as the outer layer of the Tor protocol is TLS. Some parts are colored according to their meaning: [[span(cipher suite list,style=background:skyblue)]], [[span(server name,style=background:limegreen)]], and [[span(TLS extensions,style=background:palegoldenrod)]].
|
|
|
{{{
|
|
|
#!html
|
|
|
Why use pluggable transports? It's because it's possible to fingerprint ordinary Tor traffic based on byte patterns that appear in it. Here's a hex dump of the first thing that a client sends to its entry node. It is actually a TLS [Client Hello](https://tools.ietf.org/html/rfc5246#section-7.4.1.2) message, as the outer layer of the Tor protocol is TLS. Some parts are colored according to their meaning: [[span(cipher suite list,style=background:skyblue)]], [[span(server name,style=background:limegreen)]], and [[span(TLS extensions,style=background:palegoldenrod)]].
|
|
|
```
|
|
|
<pre style="background:cornsilk">
|
|
|
00000000 16 03 01 00 e5 01 00 00 e1 03 03 d2 23 9e 55 3d ........ ....#.U=
|
|
|
00000010 21 cc 80 d5 2b 34 4b e1 be 90 04 c6 d0 f8 f9 a5 !...+4K. ........
|
... | ... | @@ -34,10 +32,9 @@ Why use pluggable transports? It's because it's possible to fingerprint ordinary |
|
|
000000D0 <span style="background:palegoldenrod">02 05 03 04 01 04 02 04 03 03 01 03 02 03 03 02</span> <span style="background:palegoldenrod">........ ........</span>
|
|
|
000000E0 <span style="background:palegoldenrod">01 02 02 02 03 00 0f 00 01 01</span> <span style="background:palegoldenrod">........ ..</span>
|
|
|
</pre>
|
|
|
}}}
|
|
|
Here's the first thing the relay sends back in response, a TLS [https://tools.ietf.org/html/rfc5246#section-7.4.1.3 Server Hello] along with a [https://tools.ietf.org/html/rfc5246#section-7.4.2 certificate] and key exchange message.
|
|
|
{{{
|
|
|
#!html
|
|
|
```
|
|
|
Here's the first thing the relay sends back in response, a TLS [Server Hello](https://tools.ietf.org/html/rfc5246#section-7.4.1.3) along with a [certificate](https://tools.ietf.org/html/rfc5246#section-7.4.2) and key exchange message.
|
|
|
```
|
|
|
<blockquote>
|
|
|
<pre style="background:lavender">
|
|
|
00000000 16 03 03 00 3e 02 00 00 3a 03 03 5c 67 84 3c 3a ....>... :..\g.<:
|
... | ... | @@ -89,12 +86,11 @@ Here's the first thing the relay sends back in response, a TLS [https://tools.ie |
|
|
000002E0 20 86 35 ee 6a 16 03 03 00 04 0e 00 00 00 .5.j... ......
|
|
|
</pre>
|
|
|
</blockquote>
|
|
|
}}}
|
|
|
```
|
|
|
|
|
|
It's easier to read these messages after they have been dissected by Wireshark. Here is the meaning of the Client Hello. The parts of the dissection that correspond to the hex dump above are colored the same way: [[span(cipher suite list,style=background:skyblue)]], [[span(server name,style=background:limegreen)]], and [[span(TLS extensions,style=background:palegoldenrod)]].
|
|
|
|
|
|
{{{
|
|
|
#!html
|
|
|
```
|
|
|
<pre style="background:cornsilk">
|
|
|
Secure Sockets Layer
|
|
|
SSL Record Layer: Handshake Protocol: Client Hello
|
... | ... | @@ -198,12 +194,11 @@ Secure Sockets Layer |
|
|
Length: 1
|
|
|
Mode: Peer allowed to send requests (1)</span>
|
|
|
</pre>
|
|
|
}}}
|
|
|
```
|
|
|
The [[span(cipher suite list,style=background:skyblue)]] is an interesting part of the message. Ticket #4744 has the story of how the Great Firewall of China blocked Tor in 2011 by looking for a distinctive cipher suite list. In response, Tor changed its cipher suite list to be the same as Firefox's. You can still tell the difference, though: The [[span(TLS extensions,style=background:palegoldenrod)]] used by Firefox are not the same as those used by Tor. For example, above there is "Elliptic curves (25 curves)", but Firefox actually has "Elliptic curves (3 curves)". You can see a lot more Client Hellos at [[meek/SampleClientHellos]]. Also notice the randomly generated server name [[span(www.kf3iammnyp.com,style=background:limegreen)]].
|
|
|
|
|
|
Here is the meaning of the Server Hello and certificate.
|
|
|
{{{
|
|
|
#!html
|
|
|
```
|
|
|
<blockquote>
|
|
|
<pre style="background:lavender">
|
|
|
Secure Sockets Layer
|
... | ... | @@ -299,21 +294,20 @@ Secure Sockets Layer |
|
|
Length: 0
|
|
|
</pre>
|
|
|
</blockquote>
|
|
|
}}}
|
|
|
The server has selected the cipher suite [[span(TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,style=background:skyblue)]]. It sends back its own [[span(TLS extensions,style=background:palegoldenrod)]]. The server has its own randomly generated [[span(server name,style=background:limegreen)]] appearing in its certificate. The names aren't a fixed byte pattern, but it's possible to fingerprint them by watching several handshakes and seeing that the names always fit the expected pattern, as [https://github.com/sethhall/bro-junk-drawer/blob/007b3a833206770bc4b85b12c39e0e01b7b998a0/detect-tor.bro this Bro script does].
|
|
|
```
|
|
|
The server has selected the cipher suite [[span(TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,style=background:skyblue)]]. It sends back its own [[span(TLS extensions,style=background:palegoldenrod)]]. The server has its own randomly generated [[span(server name,style=background:limegreen)]] appearing in its certificate. The names aren't a fixed byte pattern, but it's possible to fingerprint them by watching several handshakes and seeing that the names always fit the expected pattern, as [this Bro script does](https://github.com/sethhall/bro-junk-drawer/blob/007b3a833206770bc4b85b12c39e0e01b7b998a0/detect-tor.bro).
|
|
|
|
|
|
= obfs2 =
|
|
|
# obfs2
|
|
|
|
|
|
{{{
|
|
|
#!comment
|
|
|
```
|
|
|
Tor version 0.2.5.12 (git-3731dd5c3071dcba).
|
|
|
obfs4proxy-0.0.5
|
|
|
}}}
|
|
|
```
|
|
|
|
|
|
[https://gitweb.torproject.org/pluggable-transports/obfsproxy.git/tree/doc/obfs2/obfs2-protocol-spec.txt obfs2]
|
|
|
[obfs2](https://gitweb.torproject.org/pluggable-transports/obfsproxy.git/tree/doc/obfs2/obfs2-protocol-spec.txt)
|
|
|
was the first in the obfs family of "look-like-nothing" pluggable transports.
|
|
|
It looks like a uniformly random byte stream to a naive observer.
|
|
|
It was inspired by [https://github.com/brl/obfuscated-openssh obfuscated-openssh]
|
|
|
It was inspired by [obfuscated-openssh](https://github.com/brl/obfuscated-openssh)
|
|
|
and is succeeded by [[#obfs3|obfs3]], [[#ScrambleSuit|ScrambleSuit]], and obfs4.
|
|
|
obfs2 is deprecated these days, because it can be identified
|
|
|
by a totally passive observer,
|
... | ... | @@ -329,14 +323,13 @@ Both client and server do the same operations, |
|
|
differing only in a few constants.
|
|
|
We'll look at the client side. Here is the beginning of an obfs2 session.
|
|
|
The bytes that represent the
|
|
|
[[span(client/server ''seed'',style=background:dodgerblue)]],
|
|
|
[[span(client/server _seed_,style=background:dodgerblue)]],
|
|
|
[[span(magic number,style=background:orange)]] (encrypted),
|
|
|
[[span(padlen,style=background:gold)]] (encrypted), and
|
|
|
[[span(padding,style=background:gray)]] (encrypted)
|
|
|
are highlighted.
|
|
|
|
|
|
{{{
|
|
|
#!html
|
|
|
```
|
|
|
<blockquote>
|
|
|
<pre style="background:lavender">
|
|
|
00000000 <span style="background:dodgerblue">6a 14 a6 00 2c b0 cd bc b7 d7 a7 8f 65 c7 0b 37</span> <span style="background:dodgerblue">j...,... ....e..7</span>
|
... | ... | @@ -372,14 +365,14 @@ are highlighted. |
|
|
000000BC 2e f0 5c 7c fb 75 81 02 a5 28 98 66 63 83 dd 31 ..\|.u.. .(.fc..1
|
|
|
</pre>
|
|
|
</blockquote>
|
|
|
}}}
|
|
|
```
|
|
|
|
|
|
You can read the client seed directly from the packet trace.
|
|
|
In this case it is [[span(1cf11191affb8f3108c7a653c38410a1,style=background:dodgerblue)]].
|
|
|
From the seed we derive a key and an initial counter value for
|
|
|
[https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation#Counter_.28CTR.29 AES-CTR] mode.
|
|
|
Try it yourself using Python and the [https://www.dlitz.net/software/pycrypto/ PyCrypto] package:
|
|
|
{{{
|
|
|
[AES-CTR](https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation#Counter_.28CTR.29) mode.
|
|
|
Try it yourself using Python and the [PyCrypto](https://www.dlitz.net/software/pycrypto/) package:
|
|
|
```
|
|
|
>>> from Crypto.Cipher import AES
|
|
|
>>> from Crypto.Util import Counter
|
|
|
>>> from hashlib import sha256
|
... | ... | @@ -391,7 +384,7 @@ Try it yourself using Python and the [https://www.dlitz.net/software/pycrypto/ P |
|
|
>>> ciphertext = "e5d1ecbb726840272562bb8bc25b571a".decode("hex")
|
|
|
>>> print aes.decrypt(ciphertext).encode("hex")
|
|
|
2bf5ca7e00000030eef53458d509c177
|
|
|
}}}
|
|
|
```
|
|
|
The result of decrypting the first 16 bytes of ciphertext is
|
|
|
[[span(2bf5ca7e,style=background:orange)]][[span(00000030,style=background:gold)]][[span(eef53458d509c177,style=background:gray)]].
|
|
|
The first 4 bytes,
|
... | ... | @@ -424,7 +417,7 @@ Then they simply decrypt forever. |
|
|
Here is how it looks for the client side in the example above.
|
|
|
You can see the beginning of the Tor TLS Client Hello
|
|
|
at the beginning of the plaintext.
|
|
|
{{{
|
|
|
```
|
|
|
>>> init_seed = "1cf11191affb8f3108c7a653c38410a1".decode("hex")
|
|
|
>>> resp_seed = "6a14a6002cb0cdbcb7d7a78f65c70b37".decode("hex")
|
|
|
>>> init_secret = sha256("Initiator obfuscated data" + init_seed + resp_seed + "Initiator obfuscated data").digest()
|
... | ... | @@ -434,29 +427,28 @@ at the beginning of the plaintext. |
|
|
>>> ciphertext = "6cafc1f1c94cb825ca056047f7bad132".decode("hex")
|
|
|
>>> print aes.decrypt(ciphertext).encode("hex")
|
|
|
16030100ef010000eb0303923472ed41
|
|
|
}}}
|
|
|
```
|
|
|
|
|
|
= obfs3 =
|
|
|
# obfs3
|
|
|
|
|
|
{{{
|
|
|
#!comment
|
|
|
```
|
|
|
Tor version 0.2.5.12 (git-3731dd5c3071dcba).
|
|
|
obfs4proxy-0.0.5
|
|
|
}}}
|
|
|
```
|
|
|
|
|
|
[https://gitweb.torproject.org/pluggable-transports/obfsproxy.git/tree/doc/obfs3/obfs3-protocol-spec.txt obfs3]
|
|
|
[obfs3](https://gitweb.torproject.org/pluggable-transports/obfsproxy.git/tree/doc/obfs3/obfs3-protocol-spec.txt)
|
|
|
is another look-like-nothing transport, the successor to [[#obfs2|obfs2]].
|
|
|
Like obfs2, it is meant to look like a uniformly random byte stream.
|
|
|
It removes obfs2's vulnerability to passive identification
|
|
|
by adding a [https://en.wikipedia.org/wiki/Diffie%E2%80%93Hellman_key_exchange Diffie–Hellman key exchange].
|
|
|
by adding a [Diffie–Hellman key exchange](https://en.wikipedia.org/wiki/Diffie%E2%80%93Hellman_key_exchange).
|
|
|
While obfs2 can be identified completely passively,
|
|
|
it takes an active man-in-the-middle attack or active probing to identify obfs3 with certainty.
|
|
|
|
|
|
There is a catch with the Diffie–Hellman key exchange:
|
|
|
it has to look uniformly random like the rest of the protocol.
|
|
|
Naively, you would work in a subgroup of ℤ,,p,,^*^ for some
|
|
|
[https://en.wikipedia.org/wiki/Safe_prime safe prime number] ''p''.
|
|
|
But only about half the integers between 0 and ''p'' are in ℤ,,p,,^*^.
|
|
|
[safe prime number](https://en.wikipedia.org/wiki/Safe_prime) _p_.
|
|
|
But only about half the integers between 0 and _p_ are in ℤ,,p,,^*^.
|
|
|
A censor could watch all your key exchanges, and see that
|
|
|
the numbers you send are not really random, but are always
|
|
|
in the chosen subgroup.
|
... | ... | @@ -466,26 +458,26 @@ exchanging keys and not really sending random bytes. |
|
|
obfs3 solves this problem with an algorithm called UniformDH.
|
|
|
It's a small tweak to ordinary Diffie–Hellman.
|
|
|
It is described in
|
|
|
[https://lists.torproject.org/pipermail/tor-dev/2012-December/004245.html this mailing list post]
|
|
|
[this mailing list post](https://lists.torproject.org/pipermail/tor-dev/2012-December/004245.html)
|
|
|
from December 2012:
|
|
|
Let ''p'' = 3 mod 4 be prime, with ''q''=(''p''−1)/2 also prime, and ''p'' is at least
|
|
|
1536 bits. (2048 if there's room.) [Use [https://tools.ietf.org/html/rfc3526#section-2 group 5] or [https://tools.ietf.org/html/rfc3526#section-3 group 14]
|
|
|
from [https://tools.ietf.org/html/rfc3526 RFC 3526].] Let ''g'' be a generator of the order-''q'' subgroup of ℤ,,''p'',,^*^ (''g''=2 for
|
|
|
Let _p_ = 3 mod 4 be prime, with _q_=(_p_−1)/2 also prime, and _p_ is at least
|
|
|
1536 bits. (2048 if there's room.) [Use [group 5](https://tools.ietf.org/html/rfc3526#section-2) or [group 14](https://tools.ietf.org/html/rfc3526#section-3)
|
|
|
from [RFC 3526](https://tools.ietf.org/html/rfc3526).] Let _g_ be a generator of the order-_q_ subgroup of ℤ,,_p_,,^*^ (_g_=2 for
|
|
|
the two above groups from the RFC.)
|
|
|
|
|
|
To pick a private key, pick a random 1536-bit (or 2048-bit) number, and
|
|
|
force the low bit to 0. So there are 1535 (2047) bits of randomness,
|
|
|
which is the size of ''q''. Let ''x'' be that private key. Let ''X'' = ''g''^''x''^ mod ''p''.
|
|
|
which is the size of _q_. Let _x_ be that private key. Let _X_ = _g_^_x_^ mod _p_.
|
|
|
|
|
|
Here's the trick: When you send the public key, randomly decide to send
|
|
|
either ''X'' or ''p''−X. That will make the public key part a uniform 1536-bit
|
|
|
either _X_ or _p_−X. That will make the public key part a uniform 1536-bit
|
|
|
(2048-bit) string (well, negligibly different from uniform).
|
|
|
|
|
|
The other side constructs ''y'' and ''Y''=''g''^''y''^ mod ''p'' in the same way, and sends
|
|
|
either ''Y'' or ''p''−''Y''.
|
|
|
The other side constructs _y_ and _Y_=_g_^_y_^ mod _p_ in the same way, and sends
|
|
|
either _Y_ or _p_−_Y_.
|
|
|
|
|
|
Note that both (''p''−''Y'')^''x''^ = ''Y''^''x''^ mod ''p'' since ''x'' is even, and similarly
|
|
|
(''p''−''X'')^''y''^ = ''X''^''y''^ mod ''p'', so key derivation goes through unchanged.
|
|
|
Note that both (_p_−_Y_)^_x_^ = _Y_^_x_^ mod _p_ since _x_ is even, and similarly
|
|
|
(_p_−_X_)^_y_^ = _X_^_y_^ mod _p_, so key derivation goes through unchanged.
|
|
|
With UniformDH, the censor can't tell the difference between
|
|
|
a public key and a random 1536-bit (192-byte) byte string.
|
|
|
|
... | ... | @@ -498,8 +490,7 @@ Unlike in obfs2, the padding length is not sent as an integer; |
|
|
rather, the end of the padding is marked by a [[span(MAC on the secret,style=background:gold)]]
|
|
|
that both sides share after the key exchange.
|
|
|
|
|
|
{{{
|
|
|
#!html
|
|
|
```
|
|
|
<blockquote>
|
|
|
<pre style="background:lavender">
|
|
|
00000000 <span style="background:dodgerblue">89 5f 4c 0e 73 64 c5 dc 38 04 47 cf ec b2 a9 c4</span> <span style="background:dodgerblue">._L.sd.. 8.G.....</span>
|
... | ... | @@ -540,7 +531,7 @@ that both sides share after the key exchange. |
|
|
000000D0 <span style="background:gray">a2 f5 e8 13 ec a2 0a 7c 50 88 4a fc 40 be eb 1d</span> <span style="background:gray">.......| P.J.@...</span>
|
|
|
000000E0 <span style="background:gray">1b 8c 00 fd 24 55 aa 84 07 06 e7 d4 b0 58 04 16</span> <span style="background:gray">....$U.. .....X..</span>
|
|
|
</pre>
|
|
|
}}}
|
|
|
```
|
|
|
|
|
|
The first 192 bytes (1536 bits) from each side is its
|
|
|
[[span(public keys,style=background:dodgerblue)]].
|
... | ... | @@ -550,19 +541,19 @@ Here, the server has sent 100 bytes of padding |
|
|
and the client has sent 48.
|
|
|
|
|
|
Looking behind the scenes in this example, the client's private key (which doesn't appear on the wire) is
|
|
|
{{{
|
|
|
```
|
|
|
}}}
|
|
|
and the server's private key (which also does not appear on the wire) is
|
|
|
{{{
|
|
|
}}}
|
|
|
```
|
|
|
You can verify that these give the correct [[span(public keys,style=background:dodgerblue)]]
|
|
|
modulo the prime ''p'' from [https://tools.ietf.org/html/rfc3526#section-2 RFC 3526].
|
|
|
Both sides might have instead sent ''p''−''X'' or ''p''−''Y'',
|
|
|
modulo the prime _p_ from [RFC 3526](https://tools.ietf.org/html/rfc3526#section-2).
|
|
|
Both sides might have instead sent _p_−_X_ or _p_−_Y_,
|
|
|
but in this case by random chance they did not.
|
|
|
''X'' = 2^''x''^ mod ''p'' = 0x6213c9fd...\\
|
|
|
''Y'' = 2^''y''^ mod ''p'' = 0x895f4c0e...
|
|
|
_X_ = 2^_x_^ mod _p_ = 0x6213c9fd...\\
|
|
|
_Y_ = 2^_y_^ mod _p_ = 0x895f4c0e...
|
|
|
|
|
|
{{{
|
|
|
```
|
|
|
>>> x = 0xf525f255f6c11f0b10a1d0775638f19c7f85c5bd1628d4f903d4c0a11e585bc983f82da3515dfbe49f3828d5be0e68d1fd525c611560c788322cb911ccf77a74fdc74fa1f262dfad28ce454b93785e96fd8d4916896274e4896da990e089d14443a9242a974bbb5a2cefefcdb42c3984381e0ec71e2140826608cdd9ef52fe5be903ec5f17d91d580fd234345d7233fea29b81738dd67f712cfd56f54413d43f2889ed1cbd86781313580c13b99499aced73d36778a68b6a80137d084c7a5798
|
|
|
>>> y = 0x421e8e76020c725890488d5f45e3b035162b4ac51eb5d124fe565ff06cef6bf1b61f4a9952effa251a259587e227bdc8ffb22a92ea2e512290f5ce56063d7a39d549d2053f4496d0d5ecce4c920ef8683ef2023b1257d32607657db005e6490fb0461bfd834fb7fb8ef464ebfce014537e0c500386f2aeec7cd41a5fc5acbb9db64a49ef6a1df66a7064419eb0c7d879819f5834d679908e78b9e31fe2f444c4eefe14741c8e7c21fe811c9c0e51f094c25ef67e0cd329df611f6cb02fb56ba4
|
|
|
>>> p = 0xffffffffffffffffc90fdaa22168c234c4c6628b80dc1cd129024e088a67cc74020bbea63b139b22514a08798e3404ddef9519b3cd3a431b302b0a6df25f14374fe1356d6d51c245e485b576625e7ec6f44c42e9a637ed6b0bff5cb6f406b7edee386bfb5a899fa5ae9f24117c4b1fe649286651ece45b3dc2007cb8a163bf0598da48361c55d39a69163fa8fd24cf5f83655d23dca3ad961c62f356208552bb9ed529077096966d670c354e4abc9804f1746c08ca237327ffffffffffffffff
|
... | ... | @@ -572,16 +563,16 @@ but in this case by random chance they did not. |
|
|
'6213c9fd02880d56738b5fbf5b1b95f078e6a88e541576642c74eab8ddfab7dc86f5ef474b47e53860311ebe9b81c500ca5b7b00bbfcd7b80e0245eb737d42487924a39f9418a87a58a02391b3b2684f528ed55d69927bb5a57194970655c95490d074796764f44fd6de1c09c566cb0a67dcb584f0d820edfd03089ffae218342d8adeb19ee9c41a7784b44d01074d077a4909ac25cefbe92a3a8e0bac0f038131db48bcae8e5b3c4a76e3b87eaf6e31cf81f6cb40cba132eefdec19ae908b20'
|
|
|
>>> "%0384x" % Y
|
|
|
'895f4c0e7364c5dc380447cfecb2a9c47ea69821939612319959ffef7ef9acefe10c0f7ebc32f1daa701a0fe90af01ce34d0fe36af6470f1ad0e1719f424171f94ef0552405f98a4d0976a9f6152d27e2eb75b8316bd7bd812b3bc325f3ee321ca14b46eb58ea561d136cef0d962a74d495dbeab323054cb635298593c8ef7203739a71d4857398b6d8058e03454fd1a7fb483a64478aacc50dfe4eb5ccb613e8dd508b2d6ddda472db693e8edcdb23f740c4c7df48a5295f184590b1ff9c2b2'
|
|
|
}}}
|
|
|
```
|
|
|
|
|
|
Now both sides have the information they need to compute their shared secret
|
|
|
''X''^''y''^ = ''Y''^''x''^ = 2^''xy''^ = 0x8cb480f32c2f26e79b40080a3c5c34512c6e83fa1957088dcdffbdb6adcbf24b237faed92f0d0b0310bc6b1b5c99038f3a75400a0b8fb3e0cd6f4f11696ee7be2c7dafb0d8f390fbf3f186e847df4fab00ad784c9ac9806aa82cf06433e66a99293862768db38506bafd976b67ea4b08780689d5217f501748658081e92328c13284b8156eaf00cba0b343b096da71849507a97a4b41236125b443ff7e1e78f2b5489e3ac787d3a8a84dee442408525aba81c7b99721c94edcaa6513948a6607
|
|
|
_X_^_y_^ = _Y_^_x_^ = 2^_xy_^ = 0x8cb480f32c2f26e79b40080a3c5c34512c6e83fa1957088dcdffbdb6adcbf24b237faed92f0d0b0310bc6b1b5c99038f3a75400a0b8fb3e0cd6f4f11696ee7be2c7dafb0d8f390fbf3f186e847df4fab00ad784c9ac9806aa82cf06433e66a99293862768db38506bafd976b67ea4b08780689d5217f501748658081e92328c13284b8156eaf00cba0b343b096da71849507a97a4b41236125b443ff7e1e78f2b5489e3ac787d3a8a84dee442408525aba81c7b99721c94edcaa6513948a6607
|
|
|
This shared secret is used to derive marks that
|
|
|
indicate the end of padding,
|
|
|
as well as session keys that encrypt all the following payload data.
|
|
|
|
|
|
The end-of-padding marks are HMACs that use the shared secret:
|
|
|
{{{
|
|
|
```
|
|
|
>>> secret = ("%0384x" % pow(X, y, p)).decode("hex")
|
|
|
>>> import hmac
|
|
|
>>> from hashlib import sha256
|
... | ... | @@ -589,15 +580,14 @@ The end-of-padding marks are HMACs that use the shared secret: |
|
|
'7cbbee8bd9b26bf9099d75ec10b50b340d637316040acedd56c1b6d0715a0ba1'
|
|
|
>>> hmac.new(secret, "Responder magic", sha256).hexdigest()
|
|
|
'9c616cd7963c68e4a9322a3b390bf4a51ab0873a333943aff740da2dcab2d76c'
|
|
|
}}}
|
|
|
```
|
|
|
Now that the client has something to send, it sends another block of
|
|
|
[[span(random padding,style=background:gray)]] of between 0 and 4097 bytes, followed by its end-of-padding mark
|
|
|
[[span(7cbbee8bd9b26bf9099d75ec10b50b340d637316040acedd56c1b6d0715a0ba1,style=background:gold)]],
|
|
|
then followed by its encrypted payload data,
|
|
|
encrypted using a key derived from the shared secret.
|
|
|
|
|
|
{{{
|
|
|
#!html
|
|
|
```
|
|
|
<pre style="background:cornsilk">
|
|
|
000000F0 <span style="background:gray">84 57 5f 0d 73 a3 36 45 2e a2 8e 58 e7 a4 71 f4</span> <span style="background:gray">.W_.s.6E ...X..q.</span>
|
|
|
00000100 <span style="background:gray">ab 75 00 f1 91 7f 89 e5 7e 13 6e 94 91 16 90 78</span> <span style="background:gray">.u...... ~.n....x</span>
|
... | ... | @@ -625,15 +615,14 @@ encrypted using a key derived from the shared secret. |
|
|
00000238 14 11 60 90 f3 85 52 42 4b be 2f 0b 15 56 c4 26 ..`...RB K./..V.&
|
|
|
00000248 63 36 cf b7 c6..
|
|
|
</pre>
|
|
|
}}}
|
|
|
```
|
|
|
|
|
|
In its response, the server also sends some more
|
|
|
[[span(random padding,style=background:gray)]] and its own end-of-padding marker
|
|
|
[[span(9c616cd7963c68e4a9322a3b390bf4a51ab0873a333943aff740da2dcab2d76c,style=background:gold)]],
|
|
|
followed by encrypted payload data.
|
|
|
|
|
|
{{{
|
|
|
#!html
|
|
|
```
|
|
|
<blockquote>
|
|
|
<pre style="background:lavender">
|
|
|
00000124 <span style="background:gray">a3 ef 3e bf 1b fd 66 f6 91 e0 11 4b cc 99 4f c4</span> <span style="background:gray">..>...f. ...K..O.</span>
|
... | ... | @@ -653,7 +642,7 @@ followed by encrypted payload data. |
|
|
000001B8 22 69 39 f5 0f 02 02 0b c2 9f 5d 70 5f 97 16 5a "i9..... ..]p_..Z
|
|
|
</pre>
|
|
|
</blockquote>
|
|
|
}}}
|
|
|
```
|
|
|
|
|
|
Neither obfs2 nor obfs3 do anything to disguise packet sizes and timing.
|
|
|
In this comparison graphic, you can see that apart from the differing handshakes
|
... | ... | @@ -664,45 +653,37 @@ The darkness of each pixel indicates byte values from 0 to 255. |
|
|
All the bytes of obfs2 and obfs3 appear uniformly random,
|
|
|
while if you look closely at the ordinary Tor example,
|
|
|
you can see a dark smear at the beginning of each packet—that's
|
|
|
the plaintext [https://en.wikipedia.org/wiki/Transport_Layer_Security#TLS_record TLS record header],
|
|
|
the plaintext [TLS record header](https://en.wikipedia.org/wiki/Transport_Layer_Security#TLS_record),
|
|
|
one of the features that pluggable transports aim to obscure.
|
|
|
{{{
|
|
|
#!th align=center
|
|
|
```
|
|
|
ordinary Tor
|
|
|
}}}
|
|
|
{{{
|
|
|
#!th align=center
|
|
|
```
|
|
|
```
|
|
|
obfs2
|
|
|
}}}
|
|
|
{{{
|
|
|
#!th align=center
|
|
|
```
|
|
|
```
|
|
|
obfs3
|
|
|
}}}
|
|
|
```
|
|
|
|----------------
|
|
|
{{{
|
|
|
#!td valign=bottom
|
|
|
[[Image(plain-pixels.png)]]
|
|
|
}}}
|
|
|
{{{
|
|
|
#!td valign=bottom
|
|
|
[[Image(obfs2-pixels.png)]]
|
|
|
}}}
|
|
|
{{{
|
|
|
#!td valign=bottom
|
|
|
[[Image(obfs3-pixels.png)]]
|
|
|
}}}
|
|
|
|
|
|
= ScrambleSuit =
|
|
|
|
|
|
{{{
|
|
|
#!comment
|
|
|
```
|
|
|
![plain-pixels.png](plain-pixels.png)
|
|
|
```
|
|
|
```
|
|
|
![obfs2-pixels.png](obfs2-pixels.png)
|
|
|
```
|
|
|
```
|
|
|
![obfs3-pixels.png](obfs3-pixels.png)
|
|
|
```
|
|
|
|
|
|
# ScrambleSuit
|
|
|
|
|
|
```
|
|
|
tor 0.2.5.4-alpha-dev (git-081ff5fa83cf146a)
|
|
|
obfsproxy 0.2.9
|
|
|
}}}
|
|
|
```
|
|
|
|
|
|
[http://www.cs.kau.se/philwint/scramblesuit/ ScrambleSuit] is another transport that looks like uniform random bytes. It has a couple of twists over obfs3: It randomizes the size and timing of packets, and the server resists [http://www.cs.kau.se/philwint/static/gfc/ active probing] by requiring a secret key from the client before it will respond. The hex dump looks like random noise as you would expect.
|
|
|
{{{
|
|
|
#!html
|
|
|
[ScrambleSuit](http://www.cs.kau.se/philwint/scramblesuit/) is another transport that looks like uniform random bytes. It has a couple of twists over obfs3: It randomizes the size and timing of packets, and the server resists [active probing](http://www.cs.kau.se/philwint/static/gfc/) by requiring a secret key from the client before it will respond. The hex dump looks like random noise as you would expect.
|
|
|
```
|
|
|
<pre style="background:cornsilk">
|
|
|
00000000 00 90 a0 58 e3 05 c2 d6 f0 6e 86 5c 5d dd 57 1c ...X.... .n.\].W.
|
|
|
00000010 cf a0 ec ed 15 ed 00 e5 ed d6 d9 29 d9 31 ba 64 ........ ...).1.d
|
... | ... | @@ -737,46 +718,38 @@ obfsproxy 0.2.9 |
|
|
000000D0 75 1e 72 8f a4 f7 0c d5 6f 38 33 3e c2 58 4c d6 u.r..... o83>.XL.
|
|
|
</pre>
|
|
|
</blockquote>
|
|
|
}}}
|
|
|
```
|
|
|
|
|
|
Here are visualizations of downloading the web page at https://check.torproject.org/ with ordinary Tor, obfs3, and ScrambleSuit. obfs3 uses the same packet sizes as ordinary Tor. ScrambleSuit randomly chooses a few packet sizes, and pads packets to those sizes, so more bytes are sent overall. Differences in packet timing are not shown.
|
|
|
{{{
|
|
|
#!th align=center
|
|
|
```
|
|
|
ordinary Tor
|
|
|
}}}
|
|
|
{{{
|
|
|
#!th align=center
|
|
|
```
|
|
|
```
|
|
|
obfs3
|
|
|
}}}
|
|
|
{{{
|
|
|
#!th align=center
|
|
|
```
|
|
|
```
|
|
|
ScrambleSuit
|
|
|
}}}
|
|
|
```
|
|
|
|----------------
|
|
|
{{{
|
|
|
#!td valign=top
|
|
|
[[Image(plain-webpage.png)]]
|
|
|
}}}
|
|
|
{{{
|
|
|
#!td valign=top
|
|
|
[[Image(obfs3-webpage.png)]]
|
|
|
}}}
|
|
|
{{{
|
|
|
#!td valign=top
|
|
|
[[Image(scramblesuit-webpage.png)]]
|
|
|
}}}
|
|
|
```
|
|
|
![plain-webpage.png](plain-webpage.png)
|
|
|
```
|
|
|
```
|
|
|
![obfs3-webpage.png](obfs3-webpage.png)
|
|
|
```
|
|
|
```
|
|
|
![scramblesuit-webpage.png](scramblesuit-webpage.png)
|
|
|
```
|
|
|
|
|
|
The following two diagrams show ScrambleSuit's inter-arrival time modification. When downloading data, Tor (the blue lines) looks similar across different connections. ScrambleSuit servers (the orange lines) deviate from that. Note that inter-arrival time obfuscation is quite expensive as it artificially drains network throughput. As a result, it is disabled by default in obfsproxy.
|
|
|
{{{
|
|
|
#!td valign=top
|
|
|
[[Image(scramblesuit-inter-arrival-times.png)]]
|
|
|
}}}
|
|
|
```
|
|
|
![scramblesuit-inter-arrival-times.png](scramblesuit-inter-arrival-times.png)
|
|
|
```
|
|
|
|
|
|
= FTE =
|
|
|
# FTE
|
|
|
|
|
|
[https://fteproxy.org/ FTE], for "format-transforming encryption," encodes data so that it matches an arbitrary [https://en.wikipedia.org/wiki/Regular_expression regular expression]. The idea is that censorship hardware uses regular expressions to classify traffic into allowed and disallowed categories. FTE makes your traffic look like something in the allowed category, as far as the regular expression can tell. Here is a sample of FTE traffic using its default [https://github.com/kpdyer/fteproxy/blob/cdd30bbd441a45828965e0d62c6548a04d8bd823/fteproxy/defs/20131224.json#L2 manual-http-request] and [https://github.com/kpdyer/fteproxy/blob/cdd30bbd441a45828965e0d62c6548a04d8bd823/fteproxy/defs/20131224.json#L5 manual-http-response] formats. HTTP bodies have been truncated to save space.
|
|
|
{{{
|
|
|
#!html
|
|
|
[FTE](https://fteproxy.org/), for "format-transforming encryption," encodes data so that it matches an arbitrary [regular expression](https://en.wikipedia.org/wiki/Regular_expression). The idea is that censorship hardware uses regular expressions to classify traffic into allowed and disallowed categories. FTE makes your traffic look like something in the allowed category, as far as the regular expression can tell. Here is a sample of FTE traffic using its default [manual-http-request](https://github.com/kpdyer/fteproxy/blob/cdd30bbd441a45828965e0d62c6548a04d8bd823/fteproxy/defs/20131224.json#L2) and [manual-http-response](https://github.com/kpdyer/fteproxy/blob/cdd30bbd441a45828965e0d62c6548a04d8bd823/fteproxy/defs/20131224.json#L5) formats. HTTP bodies have been truncated to save space.
|
|
|
```
|
|
|
<pre style="background:cornsilk">
|
|
|
GET //oa9xnE79SSJT73XIDv5gDx6m9kCx.6SJzCweNTMMPPFjL/rgCK1XqYv6hSQJkzpMkpu1cTBiauAaz4Fl49NK78o2nUD/VcGRS2MM7Bfl6X4v./xGw5orrtPQfIXUbWCW.CkTS3j8sD5wQfbsURlceheKV5/bVHs3axmSbKbzvyg0dMh/xQiK2mMAR0aifZ93F0l9ql9qRSDa/8b6oZITWMZFKHwIJEFSJnrpUFj/0c9dX HTTP/1.1\r\n
|
|
|
\r\n
|
... | ... | @@ -856,17 +829,16 @@ C4\rPFt\x83v\xe5\x12\xaf\xc4\x0b\x1f\x96\xb9\xa2\xa2\xa1\x84\xee\xbb\xa0 |
|
|
...
|
|
|
</pre>
|
|
|
</blockquote>
|
|
|
}}}
|
|
|
```
|
|
|
|
|
|
FTE can do other formats as well. Here is a sample of [https://github.com/kpdyer/fteproxy/blob/cdd30bbd441a45828965e0d62c6548a04d8bd823/fteproxy/defs/20131224.json#L16 manual-ssh-request] and [https://github.com/kpdyer/fteproxy/blob/cdd30bbd441a45828965e0d62c6548a04d8bd823/fteproxy/defs/20131224.json#L19 manual-ssh-response]. This example doesn't use Tor; it was run with
|
|
|
{{{
|
|
|
FTE can do other formats as well. Here is a sample of [manual-ssh-request](https://github.com/kpdyer/fteproxy/blob/cdd30bbd441a45828965e0d62c6548a04d8bd823/fteproxy/defs/20131224.json#L16) and [manual-ssh-response](https://github.com/kpdyer/fteproxy/blob/cdd30bbd441a45828965e0d62c6548a04d8bd823/fteproxy/defs/20131224.json#L19). This example doesn't use Tor; it was run with
|
|
|
```
|
|
|
fteproxy --mode client --upstream-format manual-ssh-request --downstream-format manual-ssh-response
|
|
|
fteproxy --mode server --upstream-format manual-ssh-request --downstream-format manual-ssh-response
|
|
|
ncat -k -l -p 8081
|
|
|
ncat 127.0.0.1 8079
|
|
|
}}}
|
|
|
{{{
|
|
|
#!html
|
|
|
```
|
|
|
```
|
|
|
<pre style="background:cornsilk">
|
|
|
SSH-2.0SKT\xbb\x1d\x18\xd5\xab\x11\xc0\n\xb1UCD\x15\xbf2"r\xf0\xa6\xaf\x98
|
|
|
\xa1\xf57>!r\x81\xd8\xd9\x17E$\xdflV\xb0x\xe1\x97bG\x17\x06}\x9b\xf3\x13
|
... | ... | @@ -906,18 +878,17 @@ SSH-2.0S.\xc5\x0eI7!{7\x85@\xe5\xf2\x7f\xacAZ\xdcl\x99/9\xe1 \x90\x0b\rVv |
|
|
\x94A\xcf<Hy\x1e
|
|
|
</pre>
|
|
|
</blockquote>
|
|
|
}}}
|
|
|
```
|
|
|
|
|
|
= Flash proxy =
|
|
|
# Flash proxy
|
|
|
|
|
|
[https://crypto.stanford.edu/flashproxy/ Flash proxy] is a system of short-lived proxies, each running as a JavaScript program in a web browser. Flash proxy uses a browser technology called [https://en.wikipedia.org/wiki/WebSocket WebSocket], which is a socket-like layer on top of HTTP.
|
|
|
[Flash proxy](https://crypto.stanford.edu/flashproxy/) is a system of short-lived proxies, each running as a JavaScript program in a web browser. Flash proxy uses a browser technology called [WebSocket](https://en.wikipedia.org/wiki/WebSocket), which is a socket-like layer on top of HTTP.
|
|
|
|
|
|
Flash proxy doesn't do anything special to obscure the content that it carries. What gets sent is [[#OrdinaryTor|ordinary Tor]] TLS, wrapped inside a WebSocket layer: if you look at the WebSocket payloads, you will be able to see TLS. (There actually is one slight obfuscation, a side effect of the WebSocket protocol: WebSocket frames sent from the proxy to the client are [https://tools.ietf.org/html/rfc6455#section-5.3 xored] with a 4-byte random masking key. The key is set to [[span(00 00 00 00,style=background:coral)]] in the examples below so you can see how it looks without masking.) You can see the TLS in the payloads of the examples below. Headers like `Sec-WebSocket-Key` and `Upgrade: websocket` are part of the WebSocket handshake.
|
|
|
Flash proxy doesn't do anything special to obscure the content that it carries. What gets sent is [[#OrdinaryTor|ordinary Tor]] TLS, wrapped inside a WebSocket layer: if you look at the WebSocket payloads, you will be able to see TLS. (There actually is one slight obfuscation, a side effect of the WebSocket protocol: WebSocket frames sent from the proxy to the client are [xored](https://tools.ietf.org/html/rfc6455#section-5.3) with a 4-byte random masking key. The key is set to [[span(00 00 00 00,style=background:coral)]] in the examples below so you can see how it looks without masking.) You can see the TLS in the payloads of the examples below. Headers like `Sec-WebSocket-Key` and `Upgrade: websocket` are part of the WebSocket handshake.
|
|
|
|
|
|
Notice that, because a flash proxy connects to the client, and not the other way around, the first data that gets sent (the HTTP GET request that starts a WebSocket connection) is sent ''from'' the proxy ''to'' the client. The `HTTP/1.0 101 Switching Protocols` response is sent by the client, which in this case is actually acting as a web ''server''.
|
|
|
Notice that, because a flash proxy connects to the client, and not the other way around, the first data that gets sent (the HTTP GET request that starts a WebSocket connection) is sent _from_ the proxy _to_ the client. The `HTTP/1.0 101 Switching Protocols` response is sent by the client, which in this case is actually acting as a web _server_.
|
|
|
|
|
|
{{{
|
|
|
#!html
|
|
|
```
|
|
|
<blockquote>
|
|
|
<pre style="background:lavender">
|
|
|
GET / HTTP/1.1\r\n
|
... | ... | @@ -945,10 +916,9 @@ Connection: Upgrade\r\n |
|
|
Sec-WebSocket-Accept: tZGC3i4YfiIrPSNfbfkx+JyrLYg=\r\n
|
|
|
\r\n
|
|
|
</pre>
|
|
|
}}}
|
|
|
At this point the WebSocket handshake is finished, and the client begins to send its data, a TLS [https://tools.ietf.org/html/rfc5246#section-7.4.1.2 Client Hello] message preceded by a [[span(WebSocket frame header,style=background:coral)]]. You can also see the [[span(cipher suite list,style=background:skyblue)]], [[span(server name,style=background:limegreen)]], and [[span(TLS extensions,style=background:palegoldenrod)]], just as in the [[#OrdinaryTor|ordinary Tor]] example.
|
|
|
{{{
|
|
|
#!html
|
|
|
```
|
|
|
At this point the WebSocket handshake is finished, and the client begins to send its data, a TLS [Client Hello](https://tools.ietf.org/html/rfc5246#section-7.4.1.2) message preceded by a [[span(WebSocket frame header,style=background:coral)]]. You can also see the [[span(cipher suite list,style=background:skyblue)]], [[span(server name,style=background:limegreen)]], and [[span(TLS extensions,style=background:palegoldenrod)]], just as in the [[#OrdinaryTor|ordinary Tor]] example.
|
|
|
```
|
|
|
<pre style="background:cornsilk">
|
|
|
00000000 <span style="background:coral">82 7e 00 fe</span> 16 03 01 00 f9 01 00 00 f5 03 03 1c <span style="background:coral">.~..</span>.... ........
|
|
|
00000010 54 ec ef 3b 2e bc 2b ec e0 3b 14 ff 10 8a a5 3b T..;..+. .;.....;
|
... | ... | @@ -968,10 +938,9 @@ At this point the WebSocket handshake is finished, and the client begins to send |
|
|
000000F0 <span style="background:palegoldenrod">03 03 01 03 02 03 03 02 01 02 02 02 03 00 0f 00</span> <span style="background:palegoldenrod">........ ........</span>
|
|
|
00000100 <span style="background:palegoldenrod">01 01</span> <span style="background:palegoldenrod">..</span>
|
|
|
</pre>
|
|
|
}}}
|
|
|
The flash proxy sends back some more data, a TLS [https://tools.ietf.org/html/rfc5246#section-7.4.1.3 Server Hello] preceded by a [[span(WebSocket header,style=background:coral)]]. The Server Hello includes a certificate with domain names and expiration dates, which are visible as plain text. The WebSocket header is 4 bytes longer in this direction than in the other, because it includes the [https://tools.ietf.org/html/rfc6455#section-5.3 masking key], which is set to [[span(00 00 00 00,style=background:coral)]] for this example.
|
|
|
{{{
|
|
|
#!html
|
|
|
```
|
|
|
The flash proxy sends back some more data, a TLS [Server Hello](https://tools.ietf.org/html/rfc5246#section-7.4.1.3) preceded by a [[span(WebSocket header,style=background:coral)]]. The Server Hello includes a certificate with domain names and expiration dates, which are visible as plain text. The WebSocket header is 4 bytes longer in this direction than in the other, because it includes the [masking key](https://tools.ietf.org/html/rfc6455#section-5.3), which is set to [[span(00 00 00 00,style=background:coral)]] for this example.
|
|
|
```
|
|
|
<blockquote>
|
|
|
<pre style="background:lavender">
|
|
|
00000000 <span style="background:coral">82 fe 02 f2 00 00 00 00</span> 16 03 03 00 3e 02 00 00 <span style="background:coral">........</span> ....>...
|
... | ... | @@ -1024,16 +993,15 @@ The flash proxy sends back some more data, a TLS [https://tools.ietf.org/html/rf |
|
|
000002F0 92 16 03 03 00 04 0e 00 00 00 ........ ..
|
|
|
</pre>
|
|
|
</blockquote>
|
|
|
}}}
|
|
|
```
|
|
|
After this, the communication continues in both directions, socket-like with WebSocket framing, until the flash proxy disappears.
|
|
|
|
|
|
== Flash proxy rendezvous ==
|
|
|
## Flash proxy rendezvous
|
|
|
|
|
|
There's one more piece to flash proxy, which is how the client advertises that it needs service from a flash proxy in the first place. This process is called "rendezvous." The client must send its IP address to the flash proxy system in a way that the censor can't detect and block. There are a few ways to do it, but most often it is done by a program called [https://gitweb.torproject.org/flashproxy.git/tree/doc/flashproxy-reg-appspot.1.txt flashproxy-reg-appspot] and its helper [https://gitweb.torproject.org/flashproxy.git/tree/doc/flashproxy-reg-url.1.txt flashproxy-reg-url]. flashproxy-reg-appspot works by reflecting an HTTP request through a web app running on [https://developers.google.com/appengine/ Google App Engine], using the domain name www.google.com so the censor can't tell you are talking to App Engine, and encoding your IP address as part of the URL.
|
|
|
There's one more piece to flash proxy, which is how the client advertises that it needs service from a flash proxy in the first place. This process is called "rendezvous." The client must send its IP address to the flash proxy system in a way that the censor can't detect and block. There are a few ways to do it, but most often it is done by a program called [flashproxy-reg-appspot](https://gitweb.torproject.org/flashproxy.git/tree/doc/flashproxy-reg-appspot.1.txt) and its helper [flashproxy-reg-url](https://gitweb.torproject.org/flashproxy.git/tree/doc/flashproxy-reg-url.1.txt). flashproxy-reg-appspot works by reflecting an HTTP request through a web app running on [Google App Engine](https://developers.google.com/appengine/), using the domain name www.google.com so the censor can't tell you are talking to App Engine, and encoding your IP address as part of the URL.
|
|
|
|
|
|
When you run flashproxy-reg-appspot, the client first sends one covert HTTPS request in order to learn its own external IP address. The censor doesn't get to see the contents of these messages; they are inside HTTPS encryption. The packets will be marked with a [[span(black border,style=border:2px solid black)]] to indicate that we are looking inside the encryption.
|
|
|
{{{
|
|
|
#!html
|
|
|
```
|
|
|
<pre style="background:cornsilk;border:2px solid black">
|
|
|
GET /ip HTTP/1.1\r\n
|
|
|
Accept-Encoding: identity\r\n
|
... | ... | @@ -1055,12 +1023,11 @@ Connection: close\r\n |
|
|
192.0.2.101
|
|
|
</pre>
|
|
|
</blockquote>
|
|
|
}}}
|
|
|
```
|
|
|
The Host header [[span(fp-reg-a.appspot.com,style=background:springgreen)]] is the actual destination of the request—the web app running at that domain forwards the request to the flash proxy system—but the censor instead sees the request destined for www.google.com.
|
|
|
|
|
|
The client then sends another HTTPS request that contains an encrypted payload that will be forwarded to a flash proxy:
|
|
|
{{{
|
|
|
#!html
|
|
|
```
|
|
|
<pre style="background:cornsilk;border:2px solid black">
|
|
|
GET /reg/IiYtnmOro5k8IkFxIOljS1k8Z3dC3M0m_mM40PZo4STDY1vqn4xC6l9zAOjnr_Gw5xGBnKfbjDiyc4uaN5DxkseUAw4NUxwr6UySYYYMpssBkgRe4P5LBGlQy7B8rjzFlg8snXx2yUIbfrX2hP11XB-Tvr2po6VeNEEAiXQG48waXknztb0KBTM6qTXfNiZf3QwCW7aap-yu5IwFz6thhZ1NLwNEdp0tHn42m4sbEZzANM3sFv0kBlBn9IOWtFiwzdacjS6rXiuULhhC7rR2WuhsjVctdus8qNmUnfm22c36KPIgqyB5uDSR45pq5rHBdL_ZSsadwKQwQeW_6rohaw== HTTP/1.1\r\n
|
|
|
Accept-Encoding: identity\r\n
|
... | ... | @@ -1083,16 +1050,15 @@ Connection: close\r\n |
|
|
\r\n
|
|
|
</pre>
|
|
|
</blockquote>
|
|
|
}}}
|
|
|
```
|
|
|
The response `204 No Content` means that the web app didn't have to do any work. It merely copied the long URL it was given and sent it to the flash proxy system. That long URL is actually an encoded ciphertext. Here is what it looks like decrypted:
|
|
|
{{{
|
|
|
```
|
|
|
client=192.0.2.101%3A9000&client-transport=websocket
|
|
|
}}}
|
|
|
```
|
|
|
|
|
|
= meek =
|
|
|
# meek
|
|
|
|
|
|
{{{
|
|
|
#!comment
|
|
|
```
|
|
|
meek 0.5
|
|
|
tor 0.2.24.1-1
|
|
|
In order to get the HTTP captures I had to hack the browser extension to relay plaintext through an Ncat listener that added its own SSL.
|
... | ... | @@ -1104,13 +1070,12 @@ In components/main.js, remove the https check from requestOk and do: |
|
|
Check the port number XXXXX used by the helper, then run
|
|
|
ClientTransportPlugin meek exec ./meek-client --url=https://meek-reflect.appspot.com/ --front=www.google.com --log meek-client.log --helper 127.0.0.1:XXXXX
|
|
|
Capture localhost port 8080.
|
|
|
}}}
|
|
|
```
|
|
|
|
|
|
[[meek]] uses HTTP as a transport, and TLS to hide the contents of messages. It reflects its HTTP requests through a third party server like [https://developers.google.com/appengine/ App Engine], using a technical trick to make it look like it is talking to a different server, one that is expensive for the censor to block. It uses a browser plugin in order to camouflage its TLS fingerprint.
|
|
|
[[meek]] uses HTTP as a transport, and TLS to hide the contents of messages. It reflects its HTTP requests through a third party server like [App Engine](https://developers.google.com/appengine/), using a technical trick to make it look like it is talking to a different server, one that is expensive for the censor to block. It uses a browser plugin in order to camouflage its TLS fingerprint.
|
|
|
|
|
|
Here's the first thing a censor sees when you connect using meek. It's a TLS [https://tools.ietf.org/html/rfc5246#section-7.4.1.2 Client Hello], like in the [[#OrdinaryTor|ordinary Tor]] example, but it's different: the [[span(cipher suites,style=background:skyblue)]] are different, the [[span(extensions,style=background:palegoldenrod)]] are different, and the server name is [[span(www.google.com,style=background:limegreen)]] rather than a randomly generated Tor name. The TLS looks different because it's generated by a [[#11183|web browser extension]], not by Tor, and the packet is in fact being sent to www.google.com, not to a Tor relay. Read on for the "domain fronting" trick that causes www.google.com to send the traffic to a Tor relay behind the scenes.
|
|
|
{{{
|
|
|
#!html
|
|
|
Here's the first thing a censor sees when you connect using meek. It's a TLS [Client Hello](https://tools.ietf.org/html/rfc5246#section-7.4.1.2), like in the [[#OrdinaryTor|ordinary Tor]] example, but it's different: the [[span(cipher suites,style=background:skyblue)]] are different, the [[span(extensions,style=background:palegoldenrod)]] are different, and the server name is [[span(www.google.com,style=background:limegreen)]] rather than a randomly generated Tor name. The TLS looks different because it's generated by a [[#11183|web browser extension]], not by Tor, and the packet is in fact being sent to www.google.com, not to a Tor relay. Read on for the "domain fronting" trick that causes www.google.com to send the traffic to a Tor relay behind the scenes.
|
|
|
```
|
|
|
<pre style="background:cornsilk">
|
|
|
00000000 16 03 01 00 a9 01 00 00 a5 03 01 17 24 0d d9 fe ........ ....$...
|
|
|
00000010 72 b8 1e 89 82 f2 2f 98 8a e4 88 89 85 0f dd 1e r...../. ........
|
... | ... | @@ -1124,11 +1089,10 @@ Here's the first thing a censor sees when you connect using meek. It's a TLS [ht |
|
|
00000090 <span style="background:palegoldenrod">01 00 01 00 00 0a 00 08 00 06 00 17 00 18 00 19</span> <span style="background:palegoldenrod">........ ........</span>
|
|
|
000000A0 <span style="background:palegoldenrod">00 0b 00 02 01 00 00 23 00 00 33 74 00 00</span> <span style="background:palegoldenrod">.......# ..3t..</span>
|
|
|
</pre>
|
|
|
}}}
|
|
|
```
|
|
|
|
|
|
The [https://tools.ietf.org/html/rfc5246#section-7.4.1.3 Server Hello] reply from www.google.com is ordinary—it's exactly what you would get if you went to https://www.google.com/ in a web browser. Google has a longer certificate than the typical Tor relay.
|
|
|
{{{
|
|
|
#!html
|
|
|
The [Server Hello](https://tools.ietf.org/html/rfc5246#section-7.4.1.3) reply from www.google.com is ordinary—it's exactly what you would get if you went to https://www.google.com/ in a web browser. Google has a longer certificate than the typical Tor relay.
|
|
|
```
|
|
|
<blockquote>
|
|
|
<pre style="background:lavender">
|
|
|
00000000 16 03 01 00 5e 02 00 00 5a 03 01 53 75 b8 12 35 ....^... Z..Su..5
|
... | ... | @@ -1355,11 +1319,10 @@ The [https://tools.ietf.org/html/rfc5246#section-7.4.1.3 Server Hello] reply fro |
|
|
00000DC4 34 89 d6 6b 2f 27 15 16 03 01 00 04 0e 00 00 00 4..k/'.. ........
|
|
|
</pre>
|
|
|
</blockquote>
|
|
|
}}}
|
|
|
```
|
|
|
|
|
|
Here is the dissection of those first two messages. Notice how different the Client Hello is from the one in the [[#OrdinaryTor|ordinary Tor]] example. It [[meek#HowtolooklikebrowserHTTPS|looks like a browser]] in order to be harder to fingerprint.
|
|
|
{{{
|
|
|
#!html
|
|
|
```
|
|
|
<pre style="background:cornsilk">
|
|
|
Secure Sockets Layer
|
|
|
TLSv1 Record Layer: Handshake Protocol: Client Hello
|
... | ... | @@ -1506,21 +1469,20 @@ Secure Sockets Layer |
|
|
Handshake Protocol: Certificate
|
|
|
</pre>
|
|
|
</blockquote>
|
|
|
}}}
|
|
|
```
|
|
|
|
|
|
After the TLS handshake come some HTTPS requests and responses. Their contents are not visible to the censor, because they are under a layer of HTTPS encryption. A meek session consists of many such requests, each one carrying some data. The web browser extension will reuse the same TLS session for many requests, so it doesn't have to do the TLS handshake anew every time.
|
|
|
|
|
|
== meek transport layer ==
|
|
|
## meek transport layer
|
|
|
|
|
|
What's going on under the HTTPS layer? The Tor stream is being broken into pieces and sent as a sequence of [http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html#sec9.5 HTTP POST] requests. The Tor relay sends back its data in the response body, just as if a web page were being send in response to a POST.
|
|
|
What's going on under the HTTPS layer? The Tor stream is being broken into pieces and sent as a sequence of [HTTP POST](http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html#sec9.5) requests. The Tor relay sends back its data in the response body, just as if a web page were being send in response to a POST.
|
|
|
|
|
|
To indicate that these packets are encrypted inside HTTPS and the censor doesn't get to see them, these packets will have a [[span(black border,style=border:2px solid black)]]. App Engine and the Tor relay gets to see these bytes, but the censor does not. This fact is important, because otherwise the censor could look for fixed strings like `meek-reflect.appspot.com`.
|
|
|
|
|
|
The client sends its requests to meek-reflect.appspot.com, but meek-reflect.appspot.com merely copies the requests to the meek server running on a Tor relay. That's why it's called a reflector. We won't show the communication between the reflector and the relay, because it looks the same.
|
|
|
|
|
|
The HTTP headers will be shown as plain text, but the HTTP bodies will be shown as hex dumps. The bodies are not actually hex-encoded in reality.
|
|
|
{{{
|
|
|
#!html
|
|
|
```
|
|
|
<pre style="background:cornsilk;border:2px solid black">
|
|
|
POST / HTTP/1.1\r\n
|
|
|
Host: <span style="background:springgreen">meek-reflect.appspot.com</span>\r\n
|
... | ... | @@ -1563,16 +1525,15 @@ Connection: keep-alive\r\n |
|
|
00001f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
|
|
|
0000200 00 00 00 00 00 .....
|
|
|
</pre>
|
|
|
}}}
|
|
|
Inside the encrypted HTTPS stream, we see our familiar friends, the TLS [https://tools.ietf.org/html/rfc5246#section-7.4.1.2 Client Hello], just as in the [[#OrdinaryTor|ordinary Tor]] example. Only now, it is encoded as an HTTP request body. The TLS on the inside of the requests looks different than the TLS generated by the browser extension: The TLS on the inside comes from Tor and [http://www.openssl.org/ OpenSSL], while the TLS on the outside comes from Firefox and [https://developer.mozilla.org/en-US/docs/Mozilla/Projects/NSS NSS].
|
|
|
```
|
|
|
Inside the encrypted HTTPS stream, we see our familiar friends, the TLS [Client Hello](https://tools.ietf.org/html/rfc5246#section-7.4.1.2), just as in the [[#OrdinaryTor|ordinary Tor]] example. Only now, it is encoded as an HTTP request body. The TLS on the inside of the requests looks different than the TLS generated by the browser extension: The TLS on the inside comes from Tor and [OpenSSL](http://www.openssl.org/), while the TLS on the outside comes from Firefox and [NSS](https://developer.mozilla.org/en-US/docs/Mozilla/Projects/NSS).
|
|
|
|
|
|
Notice the value of the Host header: [[span(meek-reflect.appspot.com,style=background:springgreen)]]. This is called "domain fronting," and it is the key technical trick that meek uses to hide the true destination of its communications. Even though the server name in the outside TLS handshake was [[span(www.google.com,style=background:limegreen)]], the message is really destined for [[span(meek-reflect.appspot.com,style=background:springgreen)]]. It's the same trick used by [[#FlashProxy|flashproxy-reg-appspot]] and [[GoAgent]].
|
|
|
|
|
|
Notice the session ID string [[span(cbIzfhx1Hn+,style=background:cadetblue)]]. The meek client program randomly generates this string when it receives a new connection from Tor. The server uses this string to determine which ongoing session an incoming request belongs to.
|
|
|
|
|
|
The response to the POST is the Tor relay's [https://tools.ietf.org/html/rfc5246#section-7.4.1.3 Server Hello]. Because this is the first time the server has seen this session ID, it opens a new connection to the Tor network and associates the session ID with it. Old session IDs are deleted after a period of inactivity.
|
|
|
{{{
|
|
|
#!html
|
|
|
The response to the POST is the Tor relay's [Server Hello](https://tools.ietf.org/html/rfc5246#section-7.4.1.3). Because this is the first time the server has seen this session ID, it opens a new connection to the Tor network and associates the session ID with it. Old session IDs are deleted after a period of inactivity.
|
|
|
```
|
|
|
<blockquote>
|
|
|
<pre style="background:lavender;border:2px solid black">
|
|
|
HTTP/1.1 200 OK\r\n
|
... | ... | @@ -1631,11 +1592,10 @@ Alternate-Protocol: 443:quic\r\n |
|
|
00002e0 00 00 00 ...
|
|
|
</pre>
|
|
|
</blockquote>
|
|
|
}}}
|
|
|
```
|
|
|
|
|
|
Next comes a POST sending 0 bytes and a response also sending 0 bytes. Why? HTTP doesn't provide a long-lived connection. There's no way for the server to send data back to the client without the client first making a request. So the client must send a request every so often, even if it has nothing to send to the server. This time, the server had nothing to send back.
|
|
|
{{{
|
|
|
#!html
|
|
|
```
|
|
|
<pre style="background:cornsilk;border:2px solid black">
|
|
|
POST / HTTP/1.1\r\n
|
|
|
Host: <span style="background:springgreen">meek-reflect.appspot.com</span>\r\n
|
... | ... | @@ -1655,11 +1615,10 @@ Alternate-Protocol: 443:quic\r\n |
|
|
\r\n
|
|
|
</pre>
|
|
|
</blockquote>
|
|
|
}}}
|
|
|
```
|
|
|
|
|
|
Now the client has another small packet to send. It's 150 bytes, as you can see by the Content-Length header. It happens that this time the server also has 75 bytes to send back. That isn't necessarily always the case, and the server can send back an empty response if it has nothing to send.
|
|
|
{{{
|
|
|
#!html
|
|
|
```
|
|
|
<pre style="background:cornsilk;border:2px solid black">
|
|
|
POST / HTTP/1.1\r\n
|
|
|
Host: <span style="background:springgreen">meek-reflect.appspot.com</span>\r\n
|
... | ... | @@ -1695,23 +1654,21 @@ Alternate-Protocol: 443:quic\r\n |
|
|
0000040 72 09 59 94 99 d0 18 72 76 d4 67 r.Y....rv.g
|
|
|
</pre>
|
|
|
</blockquote>
|
|
|
}}}
|
|
|
```
|
|
|
|
|
|
= Bananaphone =
|
|
|
# Bananaphone
|
|
|
|
|
|
[https://github.com/leif/bananaphone Bananaphone] encodes a data stream as a sequence of tokens somewhat resembling natural language. Each side of the communication draws its dictionary of tokens from an input corpus.
|
|
|
[Bananaphone](https://github.com/leif/bananaphone) encodes a data stream as a sequence of tokens somewhat resembling natural language. Each side of the communication draws its dictionary of tokens from an input corpus.
|
|
|
|
|
|
There is an [https://github.com/david415/obfsproxy/tree/david-bananaphone obfsproxy branch] implementing Bananaphone as a pluggable transport. http://bananaphone.readthedocs.org/en/latest/ has a description of how to set it up. It was [https://blog.torproject.org/blog/tor-weekly-news-%E2%80%94-november-13th-2013 covered] in Tor Weekly News.
|
|
|
There is an [obfsproxy branch](https://github.com/david415/obfsproxy/tree/david-bananaphone) implementing Bananaphone as a pluggable transport. http://bananaphone.readthedocs.org/en/latest/ has a description of how to set it up. It was [covered](https://blog.torproject.org/blog/tor-weekly-news-%E2%80%94-november-13th-2013) in Tor Weekly News.
|
|
|
|
|
|
Here are samples of Bananaphone traffic, using [https://archive.org/details/ulysses04300gut Ulysses] as the input corpus. Line breaks are for presentation and do not really appear in the output.
|
|
|
Here are samples of Bananaphone traffic, using [Ulysses](https://archive.org/details/ulysses04300gut) as the input corpus. Line breaks are for presentation and do not really appear in the output.
|
|
|
|
|
|
{{{
|
|
|
#!comment
|
|
|
```
|
|
|
corpus=ulysses.txt encodingSpec=words,sha1,4 modelName=markov order=2
|
|
|
}}}
|
|
|
```
|
|
|
|
|
|
{{{
|
|
|
#!html
|
|
|
```
|
|
|
<pre style="background:cornsilk">
|
|
|
him rate us seehears brazier am. Year Mr glossy lazily changed. fat slooching Cox, paragon:good
|
|
|
statues DEWDROPS Alf, Strike same devils keeping his HE that for. grand fourth A AND wont she
|
... | ... | @@ -1748,38 +1705,34 @@ WITH bit this knows tongue. bloodiest sure bit, to and You letters down not off |
|
|
occupied late frying AND being those OF distinctly has of abode Beneath Hyacinth and MACLIR
|
|
|
ingenious January midriff the I going? to profile, had Haines. paring your told old into
|
|
|
</pre>
|
|
|
}}}
|
|
|
```
|
|
|
|
|
|
Here is a comparison of the beginning of a Tor connection with plain Tor and with Bananaphone.
|
|
|
Notice how Bananaphone has less entropy (the darkness of pixels doesn't vary as much because
|
|
|
all the bytes are ASCII) and that Bananaphone adds overhead.
|
|
|
{{{
|
|
|
#!th align=center
|
|
|
```
|
|
|
ordinary Tor
|
|
|
}}}
|
|
|
{{{
|
|
|
#!th align=center
|
|
|
```
|
|
|
```
|
|
|
Bananaphone
|
|
|
}}}
|
|
|
```
|
|
|
|----------------
|
|
|
{{{
|
|
|
#!td valign=top
|
|
|
[[Image(plain-pixels-small.png)]]
|
|
|
}}}
|
|
|
{{{
|
|
|
#!td valign=top
|
|
|
[[Image(bananaphone-pixels.png)]]
|
|
|
}}}
|
|
|
```
|
|
|
![plain-pixels-small.png](plain-pixels-small.png)
|
|
|
```
|
|
|
```
|
|
|
![bananaphone-pixels.png](bananaphone-pixels.png)
|
|
|
```
|
|
|
|
|
|
= Castle =
|
|
|
# Castle
|
|
|
|
|
|
[https://github.com/bridgar/Castle-Covert-Channel Castle] encodes messages as commands in real-time-strategy video games. It uses user-interface automation to issue commands using the real game engine.
|
|
|
[Castle](https://github.com/bridgar/Castle-Covert-Channel) encodes messages as commands in real-time-strategy video games. It uses user-interface automation to issue commands using the real game engine.
|
|
|
|
|
|
This video shows the special map layout used by the transport, and how a short message is sent. It uses the free game [http://play0ad.com/ 0 A.D.]; Castle is also adaptable to other games.
|
|
|
This video shows the special map layout used by the transport, and how a short message is sent. It uses the free game [0 A.D.](http://play0ad.com/); Castle is also adaptable to other games.
|
|
|
|
|
|
[[Image(castle-0ad.png,link=https://www.youtube.com/watch?v=lQX5HpdNZ64)]]
|
|
|
![castle-0ad.png,link=https://www.youtube.com/watch?v=lQX5HpdNZ64](castle-0ad.png)
|
|
|
|
|
|
= Others =
|
|
|
# Others
|
|
|
|
|
|
There are more transports listed at https://www.torproject.org/docs/pluggable-transports and [[PluggableTransports#ListofPluggableTransports]], though most of them have not been deployed. If you know of a good way to visualize one of them, please add it to this page.
|
|
|
|
... | ... | @@ -1789,11 +1742,11 @@ Most wanted: |
|
|
* StegoTorus
|
|
|
* Dust
|
|
|
* Code Talker Tunnel
|
|
|
* Acoustic modem output of [http://www.cs.utexas.edu/~amir/papers/FreeWave.pdf FreeWave] (reimplemented in [https://www-users.cs.umn.edu/~hopper/ccs13-cya.pdf Cover Your ACKs]). Let's hear what it sounds like!
|
|
|
* Acoustic modem output of [FreeWave](http://www.cs.utexas.edu/~amir/papers/FreeWave.pdf) (reimplemented in [Cover Your ACKs](https://www-users.cs.umn.edu/~hopper/ccs13-cya.pdf)). Let's hear what it sounds like!
|
|
|
|
|
|
= Programs =
|
|
|
# Programs
|
|
|
|
|
|
Programs used to help generate the visualizations on this page:
|
|
|
{{{
|
|
|
```
|
|
|
git clone https://www.bamsoftware.com/git/garden.git
|
|
|
}}} |
|
|
``` |