I don't think it will be too difficult to add radio buttons to allow use of a default set of bridges. Will the other non-bridge configuration options be pre-configured in torrc-defaults? Or will Tor Launcher need to configure other parameters?
I don't think it will be too difficult to add radio buttons to allow use of a default set of bridges. Will the other non-bridge configuration options be pre-configured in torrc-defaults? Or will Tor Launcher need to configure other parameters?
If you want to separate the transports, separate buttons for obfs2/obfs3 and flash proxy, then for obfs2/obfs3 you just need to turn on the Bridge obfs2 and Bridge obfs3 lines. For flash proxy, you want the single Bridge flashproxy 0.0.1.0:1 line as well as LearnCircuitBuildTimeout 0 and CircuitBuildTimeout 60.
LearnCircuitBuildTimeout and CircuitBuildTimeout need some explanation. Without these settings, a flash proxy bridge will fail almost exactly 20% of the time--Tor will refuse to use it, even if it is the only bridge configured, and even if you got a connection to a browser proxy, which is the hard part. The reason is that LearnCircuitBuildTimeout is designed to learn a timeout that excludes the lowest 20% of circuits. When you connect over flash proxy, you are connecting through someone else's browser, so you never know how fast it will be. 20% of the time it will be too slow and leave you with a broken connection. There's a little writeup of the problem in this commit message. CircuitBuildTimeout might not be necessary since we are shipping Tor 0.2.4 (see #6304 (moved)), as it's set to the default value.
That said, I think this ticket is worth doing even if it is only setting new Bridge lines and not any other settings.
LearnCircuitBuildTimeout and CircuitBuildTimeout need some explanation. Without these settings, a flash proxy bridge will fail almost exactly 20% of the time--Tor will refuse to use it, even if it is the only bridge configured, and even if you got a connection to a browser proxy, which is the hard part.
I just created #10429 (closed) which may allow us to get rid of the LearnCircuitBuildTimeout 0 workaround.
A tor-level alternative to this ticket is comment:22:ticket:5018 (don't start ClientTransportPlugins until UseBridges is set). I think I prefer this ticket, because it would make it easier to enable and disable individual transports.
Do users need to be able to see and possibly edit the list of default bridges after they choose to use them? Or can the UI be as simple as a set of radio buttons that allows for No Bridges, Use Default Bridges, and Custom Bridges?
One thing that I don't see mentioned so far is updates. Users who chose “Use Default Bridges”
should probably see their bridge list updated when a newer TBB comes with an updated set of default bridges.
mcs, brade: I don't think we should support editing the default bridge set. At least not via UI. You have it right, I think. "No Bridges", "Use Default Bridges", and "Custom Bridges".
I am thinking the most straight-forward way to provide this behavior is to create a pref branch like 'extensions.torlauncher.default_bridges', and create a number of subprefs like extensions.torlauncher.default_bridges.bridge1, extensions.torlauncher.default_bridges.bridge2, and so on to arbitrary bridgeN. That way, if the "Use Default Bridges" radiobutton is selected, Tor Launcher can just enumerate this pref branch, and enter all of the values of those prefs directly as bridge lines. Otherwise, we ignore them entirely. Does that sound like it could work?
Updating extension-overrides.js will not be a problem. But once the user chooses "Use Default Bridges" the settings will be fed to tor by Tor Launcher and thus written to torrc. The current plan is to not update torrc as part of any automated update process. That means that, if "Use Default Bridges" is active, Tor Launcher would need to replace all of the old settings with new ones (pulled from the preferences). That would work OK if the only config items that need to be set are bridge lines. But I am not sure how Tor Launcher will detect that a new set of preferences has arrived (thereby triggering a refresh of the bridge settings via the control port). Setting them every time TBB starts up would work but is kind of ugly.
I am thinking the most straight-forward way to provide this behavior is to create a pref branch like 'extensions.torlauncher.default_bridges', and create a number of subprefs like extensions.torlauncher.default_bridges.bridge1, extensions.torlauncher.default_bridges.bridge2, and so on to arbitrary bridgeN. That way, if the "Use Default Bridges" radiobutton is selected, Tor Launcher can just enumerate this pref branch, and enter all of the values of those prefs directly as bridge lines. Otherwise, we ignore them entirely. Does that sound like it could work?
Yes, that should work, with the caveats I mentioned above.
Another idea (from Kathy) is to ship a second torrc-defaults that has the default bridge settings. If someone enables the "Use Default Bridges" setting, Tor Launcher would tell tor to load the alternate torrc-defaults. But I think transitioning from one torrc-defaults to another will require a complete restart of the tor process (which is not something Tor Launcher knows how to do yet).
I think completely resetting bridge settings to use the values from the prefs every startup (and also every time that pref is selected) is a fine option. Another edge case this will cover is if the user enters manual bridges at one point and then decides to try switching to the defaults at a later date. Tor launcher should wipe their current bridge settings in that case, too.
Please review and comment on the attached screenshots. For the first-run wizard, Mark and I decided that it makes sense to insert a new panel that asks a Yes/No question. This is more consistent with the other parts of the wizard and it also allows us to introduce the concept of bridges.
I don't know if it's worth mentioning here but I have seen a couple messages on the help desk from users requesting bridges… because they could not reach some hidden services. Maybe replacing “hidden relays” by “unlisted relays” or something close enough could prevent some confusion?
I don't know if it's worth mentioning here but I have seen a couple messages on the help desk from users requesting bridges… because they could not reach some hidden services. Maybe replacing “hidden relays” by “unlisted relays” or something close enough could prevent some confusion?
Thanks. We replaced "hidden relays" with "unlisted relays" as you suggested.
We finished the UI and backend changes and committed everything:
Currently this is only on a bug10418 branch in the .../user/brade/tor-launcher.git repo.
Also, we did not add any default bridges to the preferences yet.
Next we will test with the TBB 3.x-based PT bundles. We also need to check on a couple of localization issues.
The Tor Launcher "Use Default Bridges" option seems to work well, so I think this code is ready for a PT/TBB 3.6 Tor Launcher branch. Certainly it will be good to have other developers and testers try it out soon.
One issue we have to resolve is that if a particular PT is not being used (e.g., no bridges are configured), tor emits warnings such as:
Jan 22 15:20:41.000 [warn] Pluggable transport proxy (obfs2,obfs3 execPluggableTransports/obfsproxy.bin managed) does not provide any needed transportsand will not be launched.
This is no big deal, except if Tor Launcher sees lines like that it puts a caution icon on the "Open Settings" and "Copy Tor Log to Clipboard" buttons. That might generate some undesirable traffic to the help desk.
A side note: We experienced some flakiness with flashproxy (e.g., it does not find enough remotes at first, but if I use telnet to initiate an inbound TCP connection to the flashproxy port it often wakes up and finds more remotes). But we don't have any past experience flashproxy so we do not really know what to expect.
Something to think about: It would be good if flash proxy could be disabled by default, until the user has taken the steps to enable port forwarding at FlashProxyHowto. That is, it would be nice if "enable default bridges" turned on the obfsproxy bridges, but bridge flashproxy 0.0.1.0:1 were a separate switch (or even just absent from the interface, requiring you to input the bridge address manually). The reason is that enabling flash proxy without having set up port forwarding is useless and creates a little bit of network noise: an HTTP request that appears to go to www.google.com, and several incoming connection attempts from random IP addresses.
As it stands now, the majority of flash proxy registrations are broken, because it's enabled by default in the PTTBB, but most people haven't set it up.
dcf: What if we just didn't include flashproxy in the default bridge prefs? Also, is UPNP likely to remain off or insufficient? If so, should we just provide these instructions from BridgeDB, and require that flashproxy remain a custom bridge type only?
I would like to avoid cluttering the "Use default Bridges" UI with additional configuration steps and optional instructions, if possible. I'm of the opinion that it should remain a no-config option that provides bridges for the most likely functional transport type only.
I also still don't like probing as a general matter, especially in a way that happens every single startup. If we do have to go the config route or multiple transport type route, it seems like we should make the prefbranches be extensions.torlauncher.default_bridge.type.N rather than extensions.torlauncher.default_bridge.N, and have the user pick a type and stick with it.
dcf: What if we just didn't include flashproxy in the default bridge prefs? Also, is UPNP likely to remain off or insufficient? If so, should we just provide these instructions from BridgeDB, and require that flashproxy remain a custom bridge type only?
Not including flash proxy in the default bridges is a good idea. I recommend it. Now that we know LearnCircuitBuildTimeout is not needed, the only torrc changes needed to activate flash proxy are Bridge lines, and those can be edited in Tor Launcher. We can document how to do it in FlashProxyHowto.
Putting instructions in BridgeDB may not make sense because you don't use BridgeDB with flash proxy. I guess if you think of a beginning user who doesn't know where to look, so they look at BridgeDB, it would make sense to include some instructions like "this may work for you if you turn it on." For people who don't need it, it's a lot of extra information though.
I don't know whether tor-fw-helper is going to be turned on. I don't think I have any say in that. My guess is it would greatly increase the number of people who are able to use flash proxy automatically. I'm sure there will still be a lot of people (behind two NATs, behind institutional NATs) for whom it won't work. Activating flash proxy's support for port forwarding, assuming tor-fw-helper is included in the bundle, is a matter of adding an option to the ClientTransportPlugin command line. It might be nice to build a test bundle including tor-fw-helper and ask people to test it.
dcf: What if we just didn't include flashproxy in the default bridge prefs? Also, is UPNP likely to remain off or insufficient? If so, should we just provide these instructions from BridgeDB, and require that flashproxy remain a custom bridge type only?
Not including flash proxy in the default bridges is a good idea. I recommend it. Now that we know LearnCircuitBuildTimeout is not needed, the only torrc changes needed to activate flash proxy are Bridge lines, and those can be edited in Tor Launcher. We can document how to do it in FlashProxyHowto.
Ok, this sounds fine.
Putting instructions in BridgeDB may not make sense because you don't use BridgeDB with flash proxy. I guess if you think of a beginning user who doesn't know where to look, so they look at BridgeDB, it would make sense to include some instructions like "this may work for you if you turn it on." For people who don't need it, it's a lot of extra information though.
Well, I really think BridgeDB will be heading in some form of "Pick your pluggable transport type" anyway as well, with separate landing pages for each bridge type. The "Get FlashProxy Bridges" page could just include the entirety of the FlashProxy HOWTO (but hopefully a simpler form of it once we solve the port forwarding problem).
I don't know whether tor-fw-helper is going to be turned on. I don't think I have any say in that. My guess is it would greatly increase the number of people who are able to use flash proxy automatically. I'm sure there will still be a lot of people (behind two NATs, behind institutional NATs) for whom it won't work. Activating flash proxy's support for port forwarding, assuming tor-fw-helper is included in the bundle, is a matter of adding an option to the ClientTransportPlugin command line. It might be nice to build a test bundle including tor-fw-helper and ask people to test it.
Since this decision is effectively mine, I can tell you that you do in fact have a say in that, and indeed I think you just proposed a viable plan! ;)
If tor-fw-helper actually ends up working out in a test bundle, and if we can limit its execution to people who actually try to use flashproxy, it seems like a no-brainer to me to include it in the default bundle.
mcs, brade: Ok, we also made some progress discussing this on IRC. I think we do want to break out the 'extensions.torlauncher.default_bridge.' prefbranch into 'type' subtrees, and provide another radio button to allow users to select the PT type they want. Tor Launcher would then only configure Tor with the bridge lines from that type subtree. We should also add "(recommended)" as a separate DTD entity, so we can put that localized entity text next to a specific PT type we know works for a specific localized bundle. I think radiobutton type that this "(recommended)" text appears next to should be governed by another pref (that way we can switch it during the localization bundling process).
The PT choice should be recorded in a user pref and remembered across updates, even if an update changes the default set of bridges.
If Tor then fails to connect using a selected PT type, Tor Launcher should bounce back to this PT type choice dialog and indicate to the user they should either check their Internet connection, or try a new PT type.
For the actual radiobutton text, I think we can keep it simple and use the 'type' portion of the prefbranch directly as the radiobutton text, and nothing more save for an optional "(recommended)". We won't need to localize these. Using pref names like 'extensions.torlauncher.default_bridge.ObfsProxy.1', 'extensions.torlauncher.default_bridge.FlashProxy.1', and 'extensions.torlauncher.default_bridge.ScrambleSuit.1' would mean that three radio buttons would appear: "ObfsProxy", "FlashProxy", and "ScrambleSuit".
Does this make sense? Does it sound plausible and feasible?
I think we do want to break out the 'extensions.torlauncher.default_bridge.' prefbranch into 'type' subtrees, and provide another radio button to allow users to select the PT type they want. Tor Launcher would then only configure Tor with the bridge lines from that type subtree.
I think we can make this work. Is it OK to use a dropdown menu instead of radio buttons? Mark and I mocked something up and we will attach a screenshot in a minute.
If Tor then fails to connect using a selected PT type, Tor Launcher should bounce back to this PT type choice dialog and indicate to the user they should either check their Internet connection, or try a new PT type.
We may need some changes in tor to make the above happen. When all of the configured bridges are unusable, I don't think tor generates a bootstrap failure or reports an error. This means that the Tor Launcher progress window will just sit there indefinitely without making progress and the user won't know what is going on. I thought there was an open bug about this but I cannot find it.
This looks good. asn notes though that the example bridge line in "Use Custom Bridges" is confusing. Perhaps it should say something like "bridge type address:port" without the "OR". We're going to converge on that anyway, at least for the non-Tor types.
Trac: Cc: dcf1, mcs, mrphs to dcf1, mcs, mrphs, asn
This looks good. asn notes though that the example bridge line in "Use Custom Bridges" is confusing. Perhaps it should say something like "bridge type address:port" without the "OR". We're going to converge on that anyway, at least for the non-Tor types.Replying to mikeperry:
This looks good. asn notes though that the example bridge line in "Use Custom Bridges" is confusing. Perhaps it should say something like "bridge type address:port" without the "OR". We're going to converge on that anyway, at least for the non-Tor types.
In Tor Launcher, the "bridge" part is optional and it seems simpler to omit it. We will definitely get rid of the OR.
Do we want to add a type for plain Tor bridges? Tor Launcher could just strip it out when getting/setting the configuration; eventually, if tor is modified to support the new type TL can be changed to just pass it through to tor. If the answer is "yes" we will need to decide on a good type keyword to use. Maybe just "tor"?
The other thing mentioned on IRC was the idea of using the same type keywords in the dropdown menu as we use in torrc. Should we do that? Then the dropdown would look something like:
This looks good. asn notes though that the example bridge line in "Use Custom Bridges" is confusing. Perhaps it should say something like "bridge type address:port" without the "OR". We're going to converge on that anyway, at least for the non-Tor types.Replying to mikeperry:
This looks good. asn notes though that the example bridge line in "Use Custom Bridges" is confusing. Perhaps it should say something like "bridge type address:port" without the "OR". We're going to converge on that anyway, at least for the non-Tor types.
In Tor Launcher, the "bridge" part is optional and it seems simpler to omit it. We will definitely get rid of the OR.
Do we want to add a type for plain Tor bridges? Tor Launcher could just strip it out when getting/setting the configuration; eventually, if tor is modified to support the new type TL can be changed to just pass it through to tor. If the answer is "yes" we will need to decide on a good type keyword to use. Maybe just "tor"?
The other thing mentioned on IRC was the idea of using the same type keywords in the dropdown menu as we use in torrc. Should we do that? Then the dropdown would look something like:
Yes, I think the version with the literal names ('obfs3' instead of 'ObfsProxy') is better. Since we are exposing transport names to users, let's expose the correct ones. Maybe we could do
{{{
obfs3 (obfsproxy) [recommended]
Yes, I think the version with the literal names ('obfs3' instead of 'ObfsProxy') is better. Since we are exposing transport names to users, let's expose the correct ones. Maybe we could do
{{{
obfs3 (obfsproxy) [recommended]
}}}
so that people recognize the obfsproxy name.
I don't have enough context and sense of history to know if it is worthwhile to include "obsproxy." Will many people recognize that name? If so, is "obfs3" to "obfs3 (obfsproxy)" the only special mapping we need to add to Tor Launcher or are there others?
The choice the user makes for transport type is stored in:
extensions.torlauncher.default_bridge_type
If that pref has a non-empty value, Tor Launcher uses default bridges, reconfiguring them each time TBB is started. If the stored transport type does not match any of the available default bridges, Tor Launcher will display an error message and prompt the user to choose a different type (this situation could happen after an upgrade, and we did not want to silently go from using bridges to not using any bridges).
Known issues:
Tor Launcher does not do anything special to detect the situation where Tor is unable to successfully bootstrap using the bridges that are chosen. On the other hand, if this happens while the user is still in the initial setup wizard, they will be sitting on the bridge settings wizard panel when the error occurs... which is probably where we want them to be anyway.
We did not really add any support in Tor Launcher for a pseudo-transport named "tor". Maybe we should just wait until tor itself supports that... in the meantime, you can add a set of default bridges that appear to the user (in the dropdown menu) to use a transport named tor, e.g.,
We do not add "(obfsproxy)" or other alternate names to the dropdown menu. Tor Launcher just displays the TYPE from the extensions.torlauncher.default_bridge.TYPE.# preferences.
It might be nice to build a test bundle including tor-fw-helper and ask people to test it.
Since this decision is effectively mine, I can tell you that you do in fact have a say in that, and indeed I think you just proposed a viable plan! ;)
If tor-fw-helper actually ends up working out in a test bundle, and if we can limit its execution to people who actually try to use flashproxy, it seems like a no-brainer to me to include it in the default bundle.
Here is a branch that packages tor-fw-helper on top of TBB 3.5.2. It's set up to have flashproxy-client listen on an ephemeral port and use tor-fw-helper to forward it. I sent a request to tor-qa that it be tested.
The only part that's not a no-brainer, assuming the branch actually works on all platforms, is that the safety of the miniupnpc and libnatpmp libraries is unknown (comment:3:ticket:9033). I didn't review it myself.
#11069 (moved) has been filed about the Tor PT bootstrap hang issue from comment:23.
#11071 (moved) has been filed about future work on the selection dropdown based on discussion in the dev meeting. I want something sooner rather than later, though, so I think we will likely merge this form first, and then deliberate and decide on the exact layout of #11071 (moved) later.
And to post thrice without thinking once, that nightly will not include any actual PT types or default bridges yet. I sent a mail to the PT crew asking for some bridge lines to add. Will add them as soon as I hear from them, but they might not get back to me before the nightly is built.
This appears to have a bug with how it handles the torrc. The ClientTransportPlugin lines are not being preserved. The result is that if you configure pluggable transports like obfs3, upon the second invocation the browser will fail to connect to the network. You can observe this with gk's test build for OSX, which contain working obfs3 bridge lines.
We should be preserving these ClientTransportPlugin lines regardless of the PT configuration (so that the user can go back and choose to use PTs at a later point, if necessary).
I reopened this ticket, but since this was already merged, we can open a new ticket for this if you like, but I think this does block an actual 3.6-beta release either way.
Trac: Resolution: fixed toN/A Status: closed to reopened
This appears to have a bug with how it handles the torrc. The ClientTransportPlugin lines are not being preserved. The result is that if you configure pluggable transports like obfs3, upon the second invocation the browser will fail to connect to the network. You can observe this with gk's test build for OSX, which contain working obfs3 bridge lines.
I cannot find a way to cause the ClientTransportPlugin lines to be lost. How can I reproduce the problem you saw?
The ClientTransportPlugin settings are in torrc-defaults, and Tor Launcher never does anything to modify ClientTransportPlugin configuration.
The second time I start TBB, I do see several messages like the following in the tor log:
Mar 04 09:57:54.000 [warn] We were supposed to connect to bridge '109.105.109.163:47779'using pluggable transport 'obfs3', but we can't find a pluggable transport proxysupporting 'obfs3'. This can happen if you haven't provided a ClientTransportPluginline, or if your pluggable transport proxy stopped running.
But the obfs3 process is running:
brade 3270 ... S 10:03AM 0:01.12 python PluggableTransports/obfsproxy.bin managed
And bootstrapping finishes. In fact, netstat shows outbound TCP connections to some of the configured obfs3 bridges.
I wonder if there is a race inside tor between starting the client transport plugins and trying to use the bridges? As we discussed, when default bridges are configured, Tor Launcher always starts tor with DisableNetwork=1. Then it does a SETCONF to set UseBridges=1 and to configure the bridges (Bridge=). Then it does a second SETCONF with DisableNetwork=0.
I wonder if there is a race inside tor between starting the client transport plugins and trying to use the bridges? As we discussed, when default bridges are configured, Tor Launcher always starts tor with DisableNetwork=1. Then it does a SETCONF to set UseBridges=1 and to configure the bridges (Bridge=). Then it does a second SETCONF with DisableNetwork=0.
We did a little more investigation and learned that the "can't find a pluggable transport proxy
supporting 'obfs3'" warnings are emitted by tor even if tor is started with networking enabled and if no SETCONF commands are issued by Tor Launcher.
Interestingly, if we remove cached-descriptors and cached-descriptors.new before starting TBB, those warnings are not emitted by tor.
Here are some steps to reproduce the problem just by running tor at the command line.
Run tor a second time and you will see some "can't find a pluggable transport proxy
supporting 'obfs3'" warnings. Also, netstat will probably show outbound TCP connections to some of the bridges for which warnings were emitted... which does not make a lot of sense to me. The warnings seem spurious.
rm Data/Tor/cached-descriptors and run tor a third time. The warnings will not be emitted.
When I hit the bug, bootstraping did not finish. I did get the same warnings from Tor though. The only way I could get bootstrap to finish was to remove the bundle and start it again (which would of course clear cached-descritors). I also noticed that the torrc was missing the ClientTransportPlugin lines (but not the bridge lines -- those were still there) after first run, which would be consistent with the warnings..
My mistake guys. I was looking only at torrc. torrc-defaults still has the ClientTransportPlugin lines.
I am also able to get 50% into the bootstrap process a second time, but progress hangs at that point for much longer than I'd expect, but if I wait a couple minutes, it does eventually connect (though there is still that "Open settings" warning icon due to the warn log).
If I click through the config dialog again as soon as the warning button appears, then it seems to work much more quickly.
Perhaps we are looking at a pair of Tor bugs here, but different ones than I thought.
I am also able to get 50% into the bootstrap process a second time, but progress hangs at that point for much longer than I'd expect, but if I wait a couple minutes, it does eventually connect (though there is still that "Open settings" warning icon due to the warn log).
If it stalls at 50% for almost exactly 60 seconds, then it is probably #9229 (moved). It can stall at either 20% or 50%, I think depending on the state of the cache.