meek is a [[PluggableTransports|pluggable transport]], an obfuscation layer for Tor designed to evade Internet censorship. Traffic is relayed through a third-party server that is hard to block, for example a [https://en.wikipedia.org/wiki/Content_delivery_network CDN]. It uses a trick called [https://www.bamsoftware.com/papers/fronting/ "domain fronting"] to talk to a Tor relay while appearing to talk to another domain.
* [tor-dev] A simple HTTP transport and big ideas\\
Extract and run it, and then configure these settings:
1. '''Configure''' on the first screen.
1. Check '''Tor is censored in my country''', then '''Select a built-in bridge''', then choose '''meek-amazon''' or '''meek-azure'''. If one doesn't work, try the other.
meek uses a technique called "domain fronting" to send a message to a Tor relay in a way that is hard to block.
Domain fronting is the use of different domain names at different communication layers.
The meek-client program builds a special HTTPS request and sends it to an intermediate web service with many domains behind it, such as a CDN.
What's special about the request is that one domain name (the "front domain") appears on the "outside" of the request—in the DNS query and [https://tools.ietf.org/html/rfc6066#section-3 SNI]—and a different name appears on the "inside"—in the HTTP [http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.23 Host header].
The censor sees the outside name but the CDN sees the inside name, and forwards the request to the meek-server program running on a Tor bridge.
meek-server decodes the body of the request and feeds the data into Tor.
Domain fronting and meek work when the censor blocks some domains of a web service but not all of them.
Some examples of fronting-capable services are Google, CloudFront, and [[#Webservices|others]].
See [[AChildsGardenOfPluggableTransports#meek|A Child's Garden of Pluggable Transports]] for details of how the protocol looks at the byte level, both at the TLS layer (the part visible to a censor), and at the HTTP layer (the invisible layer that carries the data).
* Browser/TorBrowser/Data/Browser/profile.default/preferences/extension-overrides.js inside Tor Browser
Copy a bridge line and change the `front=` part to another domain on the same CDN.
See [https://lists.torproject.org/pipermail/tor-talk/2015-January/036410.html What to do if meek gets blocked].
[[Image(meek-custom-front.png)]]
== Other domain fronting systems ==
meek is just one of several circumvention systems using domain fronting. You can read about the technique in general [https://www.bamsoftware.com/papers/fronting/ here].
[https://psiphon.ca/ Psiphon] uses domain fronting in some places. It has a [https://bitbucket.org/psiphon/psiphon-circumvention-system/src/319379df2677d8f711c0da127b5db735a794531c/go/?at=default fork of meek-client and meek-server ] as well as a [https://bitbucket.org/psiphon/psiphon-circumvention-system/src/319379df2677d8f711c0da127b5db735a794531c/Android/PsiphonAndroidLibrary/src/com/psiphon3/psiphonlibrary/MeekClient.java?at=default port of meek-client] to Java for Android.
[https://github.com/getlantern/flashlight Flashlight] from [https://getlantern.org/ Lantern] is an HTTP proxy that users domain fronting. [https://github.com/getlantern/enproxy enproxy] is a TCP-over-HTTP tunnel.
[https://github.com/yinghuocho/firefly-proxy FireFly Proxy] is a meek-like proxy implemented in Python. It is designed against the Great Firewall of China.
[[GoAgent]] has been used to evade the Great Firewall of China for several years. It uses domain fronting on App Engine. It is only an HTTP and HTTPS proxy; the client software sends your requested URLs to App Engine, and then the request is issued directly from the App Engine servers.
== Web services ==
Here are some web services that support domain fronting. We have been testing potential fronting services mostly through manual testing and Wget commands.
Not all of the listed services are deployed. The ones you can currently use with Tor are:
See also [https://github.com/greatfire/wiki GreatFire.org's list of mirrors], which, while they aren't using domain fronting, are hosted on the same kind of high-collateral-damage services. [https://github.com/getlantern/sitescanner sitescanner] automatically tests domain names for CloudFlare support.
[https://aws.amazon.com/cloudfront/ CloudFront] is a CDN. Your files are hosted on a generated domain name that looks like d2k1ftgv7pobq7.cloudfront.net. All these domains [https://aws.amazon.com/cloudfront/custom-ssl-domains/ support HTTPS] with a wildcard cert for *.cloudfront.net, and they can front for each other.
There is a [https://aws.amazon.com/free/ free tier], good for a year, that limits you to 50 GB per month. [https://aws.amazon.com/cloudfront/pricing/ Per-request pricing] differs by client country. Per-gigabyte costs go down the more you transfer, with a maximum of $0.19 per gigabyte. Bandwidth costs to the origin server (i.e., the Tor bridge) are lower. There's an additional cost of about $0.01 per 10,000 requests.
CloudFront allows you to use your own TLS domain name for an extra charge, but that appears to put you on a certificate with a bunch of shared SANs, which can't front for domains on different certificates.
The [https://aws.amazon.com/cloudfront/faqs/ FAQ] suggests that non-cacheable, dynamic traffic is fine.
Q. Does Amazon CloudFront support delivery of dynamic content?
Amazon CloudFront supports all files that can be served over HTTP. This includes dynamic web pages, such as HTML or PHP pages, any popular static files that are a part of your web application, such as website images, audio streams, video streams, media files or software downloads. For on-demand media files, you can also choose to stream your content using RTMP delivery. Amazon CloudFront also supports delivery of live media over HTTP.
Q. Does Amazon CloudFront cache POST responses?
Amazon CloudFront does not cache the responses to POST, PUT, DELETE, OPTIONS, and PATCH requests – these requests are proxied back to the origin server.
There's a question of what to use as the front domain. Any particular *.cloudfront.net name could be individually blockable. The generic names cloudfront.net and www.cloudfront.net don't resolve. Maybe pick one with a lot of collateral damage? Or a few, and randomly choose between them? Or connect to an IP, rather than a domain (#12208). Alexa has [http://www.alexa.com/siteinfo/cloudfront.net a list of the most popular cloudfront.net domains] ("Where do visitors go on cloudfront.net?"), which starts out:
{{{
d3dsacqprgcsqh.cloudfront.net 14.67%
deayhd4nq31b0.cloudfront.net 6.06%
d396qusza40orc.cloudfront.net 2.72%
d3v9w2rcr4yc0o.cloudfront.net 2.26%
d13yacurqjgara.cloudfront.net 2.09%
}}}
There's a [https://www.robtex.com/en/advisory/dns/net/cloudfront/d3dsacqprgcsqh/#shared_pa_ma list of CNAMES] that point to an example cloudfront.net subdomain. It appears that GFW blacklists (through DNS poisoning) *.cloudfront.net, but some names are whitelisted including d3dsacqprgcsqh.cloudfront.net and d1y9yo7q4hy8a7.cloudfront.net (9gag). GreatFire.org has a [https://en.greatfire.org/search/all/cloudfront.net list of blocked cloudfront.net subdomains].
The names [https://www.robtex.com/dns/a0.awsstatic.com.html a0.awsstatic.com], [https://www.robtex.com/dns/a1.awsstatic.com.html a1.awsstatic.com], and [https://www.robtex.com/dns/d0.awsstatic.com.html d0.awsstatic.com] look promising. I found them in the source of https://aws.amazon.com/documentation/.
Here are instructions on setting up CloudFront. Read http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/GettingStarted.html. Skip step 2, which is about uploading your files to S3 (all our data comes straight from the Tor relay). Change settings as follows:
* '''Origin Domain Name''': the domain where meek-server is running. You can use `meek.bamsoftware.com`.
* '''Origin ID''': doesn't matter.
* '''Origin SSL Protocols''': TLSv1.2 only
* '''Origin Protocol Policy''': HTTPS Only
* '''HTTPS Port''': change this if you are running meek-server on a port other than 443.
* '''Viewer Protocol Policy''': HTTPS Only
* '''Allowed HTTP Methods''': GET, HEAD, OPTIONS, PUT, POST, PATCH, DELETE. It won't work without this (specifically POST is needed).
Azure is a cloud computing platform with a [https://azure.microsoft.com/en-us/services/cdn/ CDN]. The CDN services are actually provided by Verizon or Akamai.
[https://azure.microsoft.com/en-us/pricing/details/cdn/ Pricing] is $0.09 to $0.25 per GB (depending on geographical region), getting less for higher volumes.
When you choose to use the Verizon CDN, you get a domain of the form ''subdomain''.azureedge.net, where you get to choose ''subdomain''. Formerly, you got an uncontrollable subdomain of vo.msecnd.net. ajax.aspnetcdn.com might be a good front. I've also seen azurecomcdn.net.
* [https://www.eff.org/https-everywhere/atlas/domains/msecnd.net.html HTTPS Everywhere rules for msecnd.net].
List of fronting-capable Azure domains (2017-07-24):
Here's how to set up Azure.[[Image(azure-setup.png, right)]]
* Log in at https://portal.azure.com/.
* Click "All services" in the sidebar, type "cdn" in the search box, and select "CDN profiles".
* Click "+ Add" to create a CDN profile. (You only have to do this the first time you create an endpoint.)
* Name: doesn't matter.
* Resource group: "Create new" or "Use existing" doesn't matter, I think it's just an accounting thing.
* Resouce group location: doesn't matter, only controls "where the metadata associated with the CDN profile will reside".
* Pricing tier: Standard Verizon. Standard Akamai may work too, haven't tried it.
* Select the CDN profile and click "+ Endpoint".
* Name: doesn't matter; this will become your azureedge.net subdomain.
* Origin type: Custom origin.
* Origin hostname: the domain name of your meek-server bridge.
* Origin path: ''blank''
* Origin host header: same as origin hostname.
* Uncheck HTTP, check HTTPS.
* Optimized for: General web delivery.
After about an hour, the CDN will start forwarding. However, you will get "502 Bad Gateway" errors because by default, the Azure CDN [https://social.msdn.microsoft.com/Forums/azure/en-US/a973b1ec-a39f-4ee0-a683-12dc12a3f170/akamai-not-using-sni-towards-the-endpoint?forum=azurecdn#385a3443-b853-4d24-b07a-7a48ec386859 does not use TLS SNI towards the origin]. You have to ask them to enable it. (They expect you to have a one-hostname-per-IP-address setup that doesn't require SNI.) meek-server's built-in Let's Encrypt support (`--acme-hostnames`) requires SNI. Your options are to get a certificate manually and use the `--cert` and `--key` options; or to open a support request and ask them to enable SNI, which takes about one week.
(I use --no-check-certificate because the certificate isn't trusted by Wget, but it's okay in Firefox.)
[http://www.akamai.com Akamai] is a CDN.
[https://www.eff.org/https-everywhere/atlas/domains/akamaihd.net.html HTTPS Everywhere rule for akamai.net]. I don't know what's so special about the [https://a248.e.akamai.net a248.e.akamai.net] name. For example, [https://a247.e.akamai.net/ a247] and [https://a249.e.akamai.net/ a249] exist, but the certificate they serve is only good for "*.akamaihd.net", "*.akamaihd-staging.net", and "a248.e.akamai.net". [https://web.archive.org/web/20140825080101if_/https://research.microsoft.com/en-us/um/people/ratul/akamai/freeflow.pdf The paper "Fast Internet Content Delivery with FreeFlow"] describes the structure of Akamai URLs; some of it is driven by historical use when browsers didn't send Host or SNI.
The a248.e.akamai.net name started being blocked (DNS poisoned) in China in late September 2014: https://en.greatfire.org/https/a248.e.akamai.net. (See also https://en.greatfire.org/search/all/akamai.net for all akamai.net domains.)
It might be easier and cheaper to get Akamai through a reseller. For example [https://www.liquidweb.com/services/cdn.html Liquid Web] posts a price list, $100/month for up to 1000 GB. [http://lw.rrfaae.com/2012/07/using-liquid-webs-cdn-with-wordpress-via-w3-total-cache/ This blog post] describes how to use WordPress with the Liquid Web CDN. In that example they use a custom CNAME, cdn.lw.rrfaae.com, which for me has the reverse DNS a1711.g1.akamai.net. I can grab an HTTPS version of the blog while fronting through a248.e.akamai.net:
However, Liquid Web's [https://www.liquidweb.com/about/dedicatedtos.html terms of service] prohibit proxy servers:
We do not allow proxy servers of any kind, whether for personal or business use. Files with references to any proxy or likeness thereof are prohibited.
[http://cachesimple.com/ Cache Simple] (formerly Distribution Cloud) is another Akamai reseller. Their [http://cachesimple.com/get.php?p=plans pricing] starts at $50/month for 100 GB. They want you to sign a contract with a confidentiality clause when you sign up.
Rackspace offers [https://www.rackspace.com/blog/rackspace-cloud-files-cdn-launches-ssl-delivery/ Akamai access with SSL support] through their [http://www.rackspace.com/cloud/files/ Cloud Files] service. The domain you get looks like "c186397.ssl.cf1.rackcdn.com", and you can front it through a248.e.akamai.net:
However, the CDN only works for static files hosted through Cloud Files. They [https://rackspace.uservoice.com/forums/71021-product-feedback/suggestions/1105435-add-the-origin-pull-feature-to-cloud-files don't support] the "origin pull" service we need.
[http://www.hpcloud.com/products-services/cdn HP Cloud] uses Akamai. But they have the same problem as Rackspace: it's only static files from HP Cloud Object Storage.
=== Fastly ===
[http://www.fastly.com/ Fastly] is a CDN, being used by the meek-like transports of Psiphon and Lantern. [[comment:4:ticket:12208|It apparently requires]] you to front without a SNI, only an IP, because their frontend server checks the SNI against the Host, and sends a 400 response if they don't match. Both other projects had to fork an HTTPS library to make it possible.
You can get an idea of some of their domains by looking at the certificate for https://global.ssl.fastly.net/. Shared SSL hosting appears to be on subdomains of a.ssl.fastly.net, global.ssl.fastly.net, or hosts.fastly.net.
The certificate's owner does not match hostname ‘23.235.40.130’
HTTP request sent, awaiting response... 200 OK
}}}
[http://www.fastly.com/pricing/ Pricing] is a minimum $50 per month, and $0.12–0.19 per GB for the first 10 TB per month. There's an additional charge per 10,000 requests.
=== Level 3 ===
Level 3 is a tier-1 network operator and also has a [http://www.level3.com/en/products-and-services/data-and-internet/cdn-content-delivery-network/ CDN].
[http://wiki.vps.net/vps-net-features/cdn/getting-started-with-the-cdn/ VPS.NET] is a reseller of the Level 3 CDN (formerly they had a deal with Akamai). Pricing is pay-as-you-go, not per-month; in other words we can buy a TB and not pay more until it's used up. The first TB is $35 and after that it's $250.
[http://www.cloudvps.com/openstack/cdn-acceleration/ CloudVPS] is another reseller. There's no extra charge over the normal VPS fee, but they say:
"The maximum free throughput of the CDN is 100 Megabit per second (Mbit/sec). Traffic above 100 Mbit/sec will be billed at our normal traffic pricing. Contact us if you plan to use the CDN for large amounts of traffic."
"The free CloudVPS CDN cannot be used for SSL delivery. Contact us if you want to speed up SSL traffic using the CDN."
It's not clear yet whether fronting works. I found some customer domains from http://trends.builtwith.com/websitelist/Level3-CDN, but I couldn't make them work.
Level 3 is [http://www.nytimes.com/2013/11/26/technology/a-peephole-for-the-nsa.html suspected of collaborating with the NSA], so there's that.
Level 3's CDN naming seems to revolve around the footprint.net domain. While HTTP requests do appear to be fronted, attempts to retrieve content from other hosts over SSL were unsuccessful. An example can be found with:
Warface.com's certificate is returned, but we see a Footprint originating error of "Invalid Protocol." Tried this with a few domains under Level 3, to no avail.
Despite that domain fronting seems not to work, we might be able to get the same effect from the URL structure of the secure.footprint.net domain. For example, there is a Free Weibo mirror at https://secure.footprint.net/pingfan/fw. It appears we can get a path under the secure.footprint.net domain. secure.footprint.net is currently [https://en.greatfire.org/https/secure.footprint.net DNS poisoned] by GFW. [https://secure.footprint.net/pingfan/fw doesn't work (HTTP 403) as of 2015-08-28.]
=== Netlify ===
[https://www.netlify.com/ Netlify] is a CDN and static-content host. Domain fronting appears to be supported with no configuration necessary. It would not be possible to run Tor on the Netlify infrastructure and a potential "meek-netlify" would require a backend to talk to. From the basic plan ($9/month) and up, API proxying is supported. There does not appear to be any bandwidth-based billing, only a fixed monthly cost. SSL on Netlify is a free service with certificates provisioned by Lets Encrypt.
A proof-of-concept has been built, but has not been load tested:
See whether these services support fronting or not.
==== HostGator et al.? ====
Cheap web hosts like HostGator sometimes offer shared SSL. For example HostGator [https://support.hostgator.com/articles/ssl-certificates/ssl-setup-use/how-to-set-up-and-use-your-shared-ssl puts you on a name] like secure123.hostgator.com. You can probably front through those. In this case, you would run a PHP reflector (#10984) on the web host in order to reach a relay.
==== EdgeCast? ====
GreatFire has some mirrors on EdgeCast, for example https://edgecastcdn.net/00107ED/g/.
Starting November 12, 2014, edgecastcdn.net is blocked by GFW. https://en.greatfire.org/edgecastcdn.net https://twitter.com/GreatFireChina/status/533318145118048256
== Web services that appear not to work ==
Someone tried these and it looks like the domain fronting trick doesn't work.
=== Google App Engine ===
[https://cloud.google.com/appengine/docs Google App Engine] is web application hosting on Google's infrastructure.
Google App Engine used to work for domain fronting, but [https://arstechnica.com/information-technology/2018/04/google-disables-domain-fronting-capability-used-to-evade-censors/ Google disabled it] on April 13.
=== CloudFlare ===
[https://www.cloudflare.com/ CloudFlare] is a CDN. You use your own domain name. TLS is terminated at CloudFlare's server.
There are different [https://www.cloudflare.com/plans pricing plans]. The cheapest one that supports SSL is Pro, for $20 per month. Business is $200 and Enterprise averages $5,000. There's no per-gigabyte bandwidth charge.
CloudFlare [https://news.ycombinator.com/item?id=9234367 now matches the SNI and Host header] when both exist.
CloudFlare used to work for domain fronting, but does not anymore since September 2015 (comment:2:ticket:14256).
=== DreamHost ===
DreamHost's [https://www.dreamhost.com/hosting/shared/ Shared Hosting] can easily be used as a reflector using PHP, but [http://wiki.dreamhost.com/Secure_Hosting according to their "Secure Hosting" page], they don't offer shared SSL; for SSL you have to pay for your own real cert.
They have a storage service (I'm guessing S3-like) called [https://www.dreamhost.com/cloud/storage/ Dream Objects], but it's only for static files. The URLs they give are good though, like https://objects.dreamhost.com/freeweibo/.
== How to run a meek-server (bridge) ==
- Compile the program using `go build`.
- Update your torrc file. There's a sample at [https://gitweb.torproject.org/pluggable-transports/meek.git/tree/meek-server/torrc /meek-server/torrc].
- To test your bridge on the client side, you can add a line like this to your torrc:
If you're running more than one transport, you need a separate tor process for each to avoid user counting confusion. See https://lists.torproject.org/pipermail/tor-dev/2014-September/007480.html and [#Users] for more information.
== Troubleshooting ==
If meek doesn't work and you get a log message like this:
NOTICE: Bridge at '0.0.2.0:1' isn't reachable by our firewall policy. Skipping.
then you should unset the [https://www.torproject.org/docs/tor-manual.html#ReachableAddresses ReachableAddresses] and [https://www.torproject.org/docs/tor-manual.html#FascistFirewall FascistFirewall] settings in your Tor configuration. These options don't understand the dummy addresses used in meek bridge lines. See comment:4:ticket:18611 for more information.
The notation "[[span(style=color:gray,—)]]" means meek wasn't deployed on that service in that month. The notation "[[span(style=color:gray,?)]]" marks the months after I stopped handling the invoices personally. I don’t know the costs for those months, so certain totals are marked with "+" to indicate that they are higher than what is shown. Table was automatically generated by attachment:tab-meek-costs.py and attachment:meek-costs.csv.
(Note: I have started adding the costs from 4/1/2017, and will keep this page updated monthly, srabbani@cymru.com)
|||| ||= Google =||= Amazon =||= Azure =||= total =||
* [https://www.bamsoftware.com/papers/fronting/ Blocking-resistant communication through domain fronting]: Describes the technique and the experience of deployment.
* [http://pages.cs.wisc.edu/~akella/papers/ccsfp653-wangA.pdf Seeing through Network-Protocol Obfuscation]: Builds classifiers for meek, obfs3, obfs4, and FTE, and evaluates them against large traffic traces.
* [http://crad.ict.ac.cn/EN/abstract/abstract3031.shtml Towards Measuring Unobservability in Anonymous Communication Systems] (Chinese): Evaluates certain traffic features (packet size and timing) of meek and vanilla Tor, and compares them to a known non-circumvention trace using relative entropy.
* [https://www.decoyrouting.com/ Decoy routing] is a related idea.
== Distinguishability ==
=== Barriers to indistinguishability ===
1. '''TLS ciphersuites'''\\
Look like a browser. #4744 has the story of when tor changed its ciphersuite list to look like Firefox's in 2012. tor's list of ciphers is in [https://gitweb.torproject.org/tor.git/tree/src/common/ciphers.inc?id=89c16890095d63cc6f56a378108efc3d3f063204 src/common/ciphers.inc].
2. '''TLS extensions'''\\
Look like a browser.
3. '''Packet lengths'''\\
Do something to break up fixed-length cells.
4. '''Interpacket times'''
5. '''Upstream/downstream bandwidth ratio'''
6. '''Polling interval'''\\
When we have nothing to send, we start polling at 100 ms, and increase the interval by 50% every time no data is received, up to a maximum of 5 s. The growth pattern and the fixed cap is detectable.\\
Here's what the fixed polling of 5 s looks like in the GNOME system monitor:\\
[[Image(meek-0.1-network-history.png)]]
7. '''Maximum payload lengths'''\\
Payloads are limited to 65536 bytes. During bootstrapping and bulk downloads, a lot of bodies have exactly this size.
8. '''Behavior on random drops'''\\
Suppose the censor drops every hundredth connection to https://www.google.com/. Normal web users will just refresh; meek's stream will be broken.
9. '''Number of simultaneous requests'''\\
Browsers open many parallel connections to the same server; I think meek 0.4 opens just one.
10. '''Extra latency'''\\
The latency between the client and the front domain is likely to be measurably different from the latency between the client and the real destination.
Working in our advantage is that we are likely to be transporting web traffic, so we inherit some of its traffic characteristics.
Could test with [https://github.com/cisco/joy Joy] ([https://arxiv.org/abs/1607.01639 paper]), which does classification of TLS using plaintext metadata and netflow.
=== How to look like browser HTTPS ===
We decided to use a browser extension to make all the HTTPS requests required by meek, so that the requests look just like those made by a browser. There's an extension [ticket:11183 for Firefox] (which works with Tor Browser, so it can work in the browser bundle without shipping a second browser) and one [ticket:11393 for Chrome]. The list below is a summary of a discussion that took place [https://lists.torproject.org/pipermail/tor-dev/2014-February/006266.html on the tor-dev mailing list] and on IRC.
=== Sample client hellos ===
A big list of client hellos from different applications was moved to [[meek/SampleClientHellos]].
== Style guide ==
The word "meek" should be written in lower case, even when it is the first word in a sentence.
Exception: when it is the ''last'' word in a sentence, it should be in ALL CAPS.
When printed on glossy paper, the word should be followed by a ™ symbol;
when handwritten, decisively underlined.
Exception to everything: if it is the ''n''th appearance of "meek" in a document, and ''n''
is the description number of a non-halting Turing machine, then write
"𝕸𝕰𝕰𝕶"
in honor of [http://classes.soe.ucsc.edu/cmps210/Winter11/Papers/turing-1936.pdf Alan Turing]
