The Tor Project issueshttps://gitlab.torproject.org/groups/tpo/-/issues2020-06-27T13:44:03Zhttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/trac/-/issues/9744Read managed proxy server chain from a configuration file2020-06-27T13:44:03ZDavid Fifielddcf@torproject.orgRead managed proxy server chain from a configuration fileThe server in legacy/trac#7167 is called obfs-flash, but it is not inherently tied to obfsproxy not flash proxy. The only things obfs and flash about it are that it hardcodes commands to run obfsproxy and websocket-server. So let's gener...The server in legacy/trac#7167 is called obfs-flash, but it is not inherently tied to obfsproxy not flash proxy. The only things obfs and flash about it are that it hardcodes commands to run obfsproxy and websocket-server. So let's generalize it with a configuration file. This is what I envision:
```
ServerTransportPlugin obfs3 exec obfsproxy managed
ServerTransportPlugin websocket exec websocket-server
Alias obfs3_websocket obfs3|websocket
```
What this configuration file means is, "Here is how you run the obfs3 transport, here is how you run the websocket transport, and if Tor asks you for the transport obfs3_websocket, this is the chain you should interpret it as."
The idea of using the keyword `ServerTransportPlugin` is so that you can copy a transport configuration from a `torrc` or from the transport's README--but it is not meant to be a general `torrc` parser; in particular, `ServerTransportPlugin ... proxy` won't be allowed. The `Alias` is because tor doesn't allow the pipe character in transport names; it will become unnecessary if legacy/trac#9580 is implemented.
The above proxy configuration would go along with a `torrc` like this. I'm using the name meta-proxy-server for what is now called obfs-flash-server.
```
ServerTransportPlugin obfs3_websocket exec meta-proxy-server
```
Having an external configuration file is nice, because it allows you to for example add `--log` options to the commands, without recompiling the meta-proxy.
Imagine a bigger configuration:
```
ServerTransportPlugin rot13 exec myrot13server --managed
ServerTransportPlugin chaffer exec chaffer-server
ServerTransportPlugin obfs2,obfs3 exec obfsproxy managed
ServerTransportPlugin websocket exec websocket-server
Alias rot13_websocket rot13|websocket
Alias obfs2_websocket obfs2|websocket
Alias obfs3_websocket obfs3|websocket
Alias chaffedobfs3 chaffer|obfs3
```
With its accompanying `torrc`:
```
ServerTransportPlugin obfs2_websocket,obfs3_websocket,chaffedobfs3 exec meta-proxy-server
```
You can imagine the chaffer transport, for example, as being something that changes packet sizes and timings. Then you could chain it into obfs3 to make obfs3 less fingerprintable. You could even do dumb things like obfs3-in-obfs3-in-websocket.George KadianakisGeorge Kadianakishttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/trac/-/issues/9743Think of a good name for the obfs-flash meta-proxy and create a repo for it2020-06-27T13:44:03ZDavid Fifielddcf@torproject.orgThink of a good name for the obfs-flash meta-proxy and create a repo for itIn legacy/trac#7167 we are working on programs that chain managed proxies by manipulating their environment. The project is is currently called obfs-flash and has a repo at https://www.bamsoftware.com/git/obfs-flash.git.
Nothing in the ...In legacy/trac#7167 we are working on programs that chain managed proxies by manipulating their environment. The project is is currently called obfs-flash and has a repo at https://www.bamsoftware.com/git/obfs-flash.git.
Nothing in the design is inherently tied to obfsproxy and flash proxy, and we could run other managed transports in the same way. So it would be nice to have a name for it other than obfs-flash. It would also be nice to be using a torproject.org repo.https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/trac/-/issues/9742Give the Go pluggable transports library its own home2020-06-27T13:44:03ZDavid Fifielddcf@torproject.orgGive the Go pluggable transports library its own homeThe library that currently lives at https://gitweb.torproject.org/flashproxy.git/tree/HEAD:/websocket-transport/src/pt is nice, and is already being used by a separate program, the super-proxy server in legacy/trac#7167. It should have i...The library that currently lives at https://gitweb.torproject.org/flashproxy.git/tree/HEAD:/websocket-transport/src/pt is nice, and is already being used by a separate program, the super-proxy server in legacy/trac#7167. It should have its own home separate from the flashproxy tree, like pyptlib is separate from obfsproxy.George KadianakisGeorge Kadianakishttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/trac/-/issues/9732Define SIGTERM behavior in pluggable transports spec2020-06-27T13:44:03ZDavid Fifielddcf@torproject.orgDefine SIGTERM behavior in pluggable transports spechttps://gitweb.torproject.org/torspec.git/blob/2f031167c120761fa2a3ef3e1dc62e630ab7372a:/proposals/180-pluggable-transport.txt says:
Proxies should respond to a single INT signal by closing their listener ports and not accepting any ne...https://gitweb.torproject.org/torspec.git/blob/2f031167c120761fa2a3ef3e1dc62e630ab7372a:/proposals/180-pluggable-transport.txt says:
Proxies should respond to a single INT signal by closing their listener ports and not accepting any new connections, but keeping all connections open, then terminating when connections are all closed. Proxies should respond to a second INT signal by shutting down cleanly.
But it's silent on SIGTERM. When I `/etc/init.d/tor stop` on Debian, `torctl` kills the tor process with SIGTERM. I noticed because the super-proxy from comment:32:ticket:7167 starts subprocesses. It was properly killing the subprocesses on SIGINT, but not on SIGTERM.
My proposed wording:
Proxies should respond to a single TERM signal by closing their listener ports, closing all existing connections, and terminating.George KadianakisGeorge Kadianakishttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/trac/-/issues/9485Cleanup of pyptlib internals2020-06-27T13:44:03ZXimin LuoCleanup of pyptlib internalsThe current code has a few issues:
- parses config for every method call
- uses implicit global state (awkward to test)
- writes unconditionally to stdout (awkward to test)
- mixes config logic with write/decision logic
- inconsistent me...The current code has a few issues:
- parses config for every method call
- uses implicit global state (awkward to test)
- writes unconditionally to stdout (awkward to test)
- mixes config logic with write/decision logic
- inconsistent method names (writeMethod vs reportSuccess)
My refactoring deals with these issues. Currently it preserves the current API but deprecates it in favour of the new API and documents an easy transition path.
https://github.com/infinity0/pyptlib/compare/master...api-config
Old API (still supported; marked deprecated):
```
pyptlib.client.init(transports)
pyptlib.client.reportSuccess(...)
pyptlib.client.reportEnd()
```
New API:
```
client = pyptlib.client.ClientTransportPlugin.fromEnv()
client.init(transports)
client.reportMethodSuccess(...)
client.reportMethodsEnd()
```
Similar sort of thing for the server.George KadianakisGeorge Kadianakishttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/trac/-/issues/9411pyptlib.server.reportSuccess documentation misses required options parameter2020-06-27T13:44:04ZDavid Fifielddcf@torproject.orgpyptlib.server.reportSuccess documentation misses required options parameterhttps://pyptlib.readthedocs.org/en/master/API.html#report-results-back-to-tor has this example:
```
pyptlib.server.reportSuccess('rot13', ('127.0.0.1', 42042))
```
But this doesn't work, because of a required third `options` parameter:
`...https://pyptlib.readthedocs.org/en/master/API.html#report-results-back-to-tor has this example:
```
pyptlib.server.reportSuccess('rot13', ('127.0.0.1', 42042))
```
But this doesn't work, because of a required third `options` parameter:
```
TypeError: reportSuccess() takes exactly 3 arguments (2 given)
```
So `reportSuccess` should be documented to need `None` as a third argument if you don't need any options, or `options` should be made an optional parameter.George KadianakisGeorge Kadianakishttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/trac/-/issues/9410pyptlib.util.checkClientMode prints ENV-ERROR without raising EnvError2020-06-27T13:44:04ZDavid Fifielddcf@torproject.orgpyptlib.util.checkClientMode prints ENV-ERROR without raising EnvErrorYou get two ENV-ERROR lines on stdout when you use `pyptlib.util.checkClientMode` and `pyptlib.server.init` to try to implement a server-only transport, and no environment variables are set:
```
if pyptlib.util.checkClientMode():
pr...You get two ENV-ERROR lines on stdout when you use `pyptlib.util.checkClientMode` and `pyptlib.server.init` to try to implement a server-only transport, and no environment variables are set:
```
if pyptlib.util.checkClientMode():
print >> sys.stderr, "This is only a server."
sys.exit(1)
try:
info = pyptlib.server.init(["transport"])
except pyptlib.config.EnvError, e:
print >> sys.stderr, "pyptlib.server.init: %s" % e
sys.exit(1)
```
The output of this is:
```
ENV-ERROR Missing environment variable TOR_PT_STATE_LOCATION
ENV-ERROR Missing environment variable TOR_PT_STATE_LOCATION
pyptlib.server.init: Missing environment variable TOR_PT_STATE_LOCATION
```
I guess if `checkClientMode` is going to be checking environment variables, I would like to catch `EnvError` when I call that function.George KadianakisGeorge Kadianakishttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/trac/-/issues/9376subprocess management module for pyptlib2020-06-27T13:44:04ZXimin Luosubprocess management module for pyptlibDeveloped as part of work on the obfs/flashproxy combined plugin, but can be useful for other plugins. (The concept is general enough to go into python proper but that process will take much more time.)
https://github.com/infinity0/pypt...Developed as part of work on the obfs/flashproxy combined plugin, but can be useful for other plugins. (The concept is general enough to go into python proper but that process will take much more time.)
https://github.com/infinity0/pyptlib/compare/upstream...masterGeorge KadianakisGeorge Kadianakishttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/trac/-/issues/9330Pluggable Transports on windows are killed with TerminateProcess2020-06-27T13:44:04ZYawning AngelPluggable Transports on windows are killed with TerminateProcessThe pluggable transport that I am writing needs to do cleanup on shutdown (It write files to the pt_state directory, and use a child process).
The way that tor terminates pluggable transports under Windows currently is via ProcessTermin...The pluggable transport that I am writing needs to do cleanup on shutdown (It write files to the pt_state directory, and use a child process).
The way that tor terminates pluggable transports under Windows currently is via ProcessTerminate (common/util.c:tor_terminate_process), which is the rough windows equivalent of SIGKILL (Immediate termination, no child processes killed, no further code in the application gets executed).
In theory Tor is supposed to send SIGINT for a graceful shutdown but it doesn't appear to be doing this on Windows (Does it do this at all? obfsproxy doesn't appear to install a SIGINT handler to handle graceful teardown per the PT spec), and furthermore Microsoft's documentation hints at horrible evil happening if SIGINT is actually used (http://msdn.microsoft.com/en-us/library/xdkz3x12%28v=vs.110%29.aspx).
Some way to properly handle graceful shutdown that works across all platforms would be nice. Apparently the way Real Windows Apps approach this problem is with GenerateConsoleCtrlEvent or with PostMessage, neither which are portable.
Note: I don't do Windows development as a general rule so I may be missing something obvious.George KadianakisGeorge Kadianakishttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/trac/-/issues/9228Create an SSH pluggable transport2021-07-29T15:19:24ZGeorge KadianakisCreate an SSH pluggable transportYawning is writing an SSH pluggable transport.
It needs legacy/trac#9162 and legacy/trac#9163 to be deployed without ugly hotfixes.
This ticket tracks its progress.Yawning is writing an SSH pluggable transport.
It needs legacy/trac#9162 and legacy/trac#9163 to be deployed without ugly hotfixes.
This ticket tracks its progress.https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/trac/-/issues/9192Create a git pluggable transport2021-07-29T15:15:23ZGeorge KadianakisCreate a git pluggable transportFor the past few weeks, some lads are working on a git pluggable transport as a module to obfsproxy.
This is a ticket to "trac" its progress.For the past few weeks, some lads are working on a git pluggable transport as a module to obfsproxy.
This is a ticket to "trac" its progress.https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/trac/-/issues/9125Allow reporting different obfs ports to bridge db2021-11-08T19:54:35ZTracAllow reporting different obfs ports to bridge db`ServerTransportListenAddr` needs to be extended to allow listen on one port and announce to bridge database another because in most cases tor can not listen on low ports.
Something like NoAdvertise and NoListen ORPort flags.
**Trac**:...`ServerTransportListenAddr` needs to be extended to allow listen on one port and announce to bridge database another because in most cases tor can not listen on low ports.
Something like NoAdvertise and NoListen ORPort flags.
**Trac**:
**Username**: hsnhttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/trac/-/issues/9022Create an XMPP pluggable transport2021-07-29T15:18:46ZGeorge KadianakisCreate an XMPP pluggable transportWe should look into XMPP pluggable transports. There are many public XMPP services that see widespread use even from censored countries.We should look into XMPP pluggable transports. There are many public XMPP services that see widespread use even from censored countries.https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/trac/-/issues/9011Move third-party pluggable transports licenses to /pluggable-transports/bundl...2020-06-27T13:44:05ZDavid Fifielddcf@torproject.orgMove third-party pluggable transports licenses to /pluggable-transports/bundle.gitThere is a new home for the PT bundle-building scripts that formerly lived in `/flashproxy/doc`.
https://gitweb.torproject.org/pluggable-transports/bundle.git
Let's move the licenses for M2Crypto, PyCrypto, zope, etc., etc., there (and...There is a new home for the PT bundle-building scripts that formerly lived in `/flashproxy/doc`.
https://gitweb.torproject.org/pluggable-transports/bundle.git
Let's move the licenses for M2Crypto, PyCrypto, zope, etc., etc., there (and change the makefile to match). These bundle scripts are where we're actually integrating and distributing all the third-party software, so it makes sense to keep the licenses there. Someone receiving a copy of e.g. obfsproxy doesn't need the e.g. zope license--that's only necessary in the bundle where we actually distribute zope.
The `LICENSE` files of flash proxy and obfsproxy can go back to only containing the licenses of flash proxy and obfsproxy.George KadianakisGeorge Kadianakishttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/trac/-/issues/8942Update gnulinux pluggable transports bundle instructions for Debian 72020-06-27T13:44:05ZDavid Fifielddcf@torproject.orgUpdate gnulinux pluggable transports bundle instructions for Debian 7https://gitweb.torproject.org/flashproxy.git/blob/HEAD:/doc/bundle-gnulinux.txthttps://gitweb.torproject.org/flashproxy.git/blob/HEAD:/doc/bundle-gnulinux.txtGeorge KadianakisGeorge Kadianakishttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/trac/-/issues/8676Research an HTTP pluggable transport that actually uses a browser and a web s...2021-11-08T19:54:05ZGeorge KadianakisResearch an HTTP pluggable transport that actually uses a browser and a web serverResearch like www.cs.utexas.edu/~shmat/shmat_oak13parrot.pdf makes it even more clear that it's worth exploring the possibility of actually using the software you are trying to emulate. That is, if you are trying to look like Skype, you ...Research like www.cs.utexas.edu/~shmat/shmat_oak13parrot.pdf makes it even more clear that it's worth exploring the possibility of actually using the software you are trying to emulate. That is, if you are trying to look like Skype, you better use the Skype binary. If you want to look like HTTP, you better use a browser on the client-side and a web server on the server-side.
We should look whether we can use stuff like Webkit to write a client-side transport, and a web server like nginx or apache to write its server-side.https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/trac/-/issues/8645Pluggable transports bundles warn of need to upgrade when no new version is y...2020-06-27T13:44:05ZDavid Fifielddcf@torproject.orgPluggable transports bundles warn of need to upgrade when no new version is yet availableSay there is a TBB and corresponding PT bundle both with version number X.Y.Z-alpha-1. https://check.torproject.org/RecommendedTBBVersions will say:
```
[
/* ... stable bundles ... */
"X.Y.Z-alpha-1-MacOS",
"X.Y.Z-alpha-1-Windows",
"X.Y....Say there is a TBB and corresponding PT bundle both with version number X.Y.Z-alpha-1. https://check.torproject.org/RecommendedTBBVersions will say:
```
[
/* ... stable bundles ... */
"X.Y.Z-alpha-1-MacOS",
"X.Y.Z-alpha-1-Windows",
"X.Y.Z-alpha-1-Linux"
]
```
When the X.Y.Z-alpha-2 TBB is released, the file changes to
```
[
/* ... stable bundles ... */
"X.Y.Z-alpha-2-MacOS",
"X.Y.Z-alpha-2-Windows",
"X.Y.Z-alpha-2-Linux"
]
```
If we haven't built new corresponding PT bundles yet, those will still have the old version number X.Y.Z-alpha-1, and users will get a blinking Tor button telling them to upgrade. However no upgrade exists for them yet.George KadianakisGeorge Kadianakishttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/trac/-/issues/8644Pluggable transports bundles need an independent version number2020-06-27T13:44:05ZDavid Fifielddcf@torproject.orgPluggable transports bundles need an independent version numberThe PT build instructions say to increment the alpha number whenever you make a release using the same upstream plain TBB. This leads to the situation where our second batch of bundles (legacy/trac#8549) built from plain 2.4.11-alpha-1 w...The PT build instructions say to increment the alpha number whenever you make a release using the same upstream plain TBB. This leads to the situation where our second batch of bundles (legacy/trac#8549) built from plain 2.4.11-alpha-1 where called 2.4.11-alpha-2. In the meantime, another plain TBB was released, also called 2.4.11-alpha-2.
I propose we append another "Release" number in the same way that RPM packages do:
https://fedoraproject.org/wiki/Packaging:NamingGuidelines#Release_Tag
In this scheme, our first bundles built from 2.4.11-alpha-1 would be called 2.4.11-alpha-1-1, and the next bundles built from 2.4.11-alpha-1 would be 2.4.11-alpha-1-2.
I wonder if we should make it a little more clear that the "Release" number belongs to PT bundles using numbers like 2.4.11-alpha-1-pt1 and 2.4.11-alpha-1-pt2.George KadianakisGeorge Kadianakishttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/trac/-/issues/8416Programmatize PT bundle building2020-06-27T13:44:05ZDavid Fifielddcf@torproject.orgProgrammatize PT bundle buildingWe have good instructions for taking a TBB and turning it into a PT bundle:
https://gitweb.torproject.org/flashproxy.git/blob/HEAD:/doc/bundle-gnulinux.txt
https://gitweb.torproject.org/flashproxy.git/blob/HEAD:/doc/bundle-macosx.txt
ht...We have good instructions for taking a TBB and turning it into a PT bundle:
https://gitweb.torproject.org/flashproxy.git/blob/HEAD:/doc/bundle-gnulinux.txt
https://gitweb.torproject.org/flashproxy.git/blob/HEAD:/doc/bundle-macosx.txt
https://gitweb.torproject.org/flashproxy.git/blob/HEAD:/doc/bundle-windows.txt
But the instructions are manual and tedious to follow. They should rather be shell scripts.
The scripts can assume all the necessary components are already in place in some well-known locations. The `torrc` fragments need to be broken out into separate files.
The PT transport bundle is wider than flash proxy, so the build scripts don't really belong in the flash proxy repository. But they can go there for now.David Fifielddcf@torproject.orgDavid Fifielddcf@torproject.orghttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/trac/-/issues/8114Dead links from projects/obfsproxy.html.en to flashproxy bundles2020-06-27T13:44:06ZGeorge KadianakisDead links from projects/obfsproxy.html.en to flashproxy bundlesSeems like that the internal website git links (`<version-torobfsbundlealpha>` etc.) changed to `2.4.9`, and that broke all the links to the bundles in https://www.torproject.org/projects/obfsproxy.html.en .
Maybe we should hardcode the...Seems like that the internal website git links (`<version-torobfsbundlealpha>` etc.) changed to `2.4.9`, and that broke all the links to the bundles in https://www.torproject.org/projects/obfsproxy.html.en .
Maybe we should hardcode the links to those bundles to `2.4.7` for now, till they get into the regular bundle release schedule?George KadianakisGeorge Kadianakis