Design new flow to circumvent censorship

changed milestone to %Sponsor 96: Rapid Expansion of Access to the Uncensored Internet through Tor in China, Hong Kong, & Tibet

added Doing UX Team S96-maybe-O3 UX labels

assigned to @duncan

changed the description

mentioned in issue #40780 (closed)

mentioned in issue #40773 (closed)

mentioned in issue #40782 (closed)

The user flows and design specification themselves haven't changed since they were last posted in #29590 (closed), however the UI has evolved significantly from the wireframes included in that ticket:

User flows

Connection Assist

Notes:

Inherits the changes made to the browser chrome, layout and Configure Connection... button detailed in #40780 (closed).
Introduces a breadcrumb system to help set user expectations, provide a sense of progress, geography and house the various error screens that may result should our attempts to automatically apply censorship circumvention settings fail. Users may also advance to the second step in the process to apply their location settings manually instead (see "Location Settings" below).
Renders the error string in a regular paragraph within the body content.
Updates Copy Tor Logs to View logs..., with the intention that it mimics the same behavior currently exhibited in about:preferences#tor by opening a modal, partially solving both #40553 (closed) and #40544 (closed).

Location Settings

Notes:

Allows the user to switch between the first and second steps in the breadcrumb before trying a bridge.
Shifts Configure Connection... into a text link the the body content to accommodate the location settings UI below.
Reveals a location dropdown with a full ISO country list (with censored locations pinned to the top of the list in a "Frequently selected" section). The button is currently deactivated and set to 40% opacity until a country has been selected, although it could be changed to feature "Automatic" as the default option to match the same button in #40782 (closed) instead.

Trying a Bridge

Notes:

Behaves similarly to the vanilla Connecting... screen as visualized in #40780 (closed), with the important exception that it cycles through the breadcrumb steps above as the user's location is determined and circumvention settings applied.

Bridge error

Notes:

Should the user's location be set to automatic (i.e. they did not provide their location manually) and Tor Browser has failed to reach the network after applying the corresponding circumvention settings, the resulting error should check that the location lookup was correct. Users will be able to override the automatically set location using the dropdown selected below, and initiate a second connection attempt.

Last attempt

Notes:

Heading string updates from "Trying a bridge..." to "Trying one more time...".

Final error

Notes:

In the event Connection Assist fails to connect to the network after the troubleshooting methods above have been exhausted, the final error includes a link to the tb-manual (in the event we bundle it into the browser as per #31539 (closed), thereby also solving the issue of dead learn more links within #40782 (closed)).
Shits Configure Connection... as the recommended action, in contrast to previous screens.
May include an additional secondary action if we think it would be helpful (I've included Restart Tor Browser in the mockup, but I'm open to input here on whether that would be useful at all).

marked this issue as related to #40773 (closed)

mentioned in issue #29590 (closed)

mentioned in issue #40553 (closed)

mentioned in issue #40544 (closed)

mentioned in issue tpo/ux/research#20 (closed)

marked this issue as related to tpo/ux/research#20 (closed)

So we have a slight issue on this screen if bootstrap fails but not due to censorship:

<richard> hi, i'm bob and i work at big tech company
<richard> at work I need to use a proxy to get out of the corporate network
<richard> so i have that set on my laptop
<richard> i come home and said proxy is not accessible on my local network
<richard> first bootstrap fails because of proxy error
<richard> but now i'm given only censorship circumvnetion screens
<richard> when really i need a connect button after disabling my proxy
<richard> (i think just making the connect button visible but not the primary button on the first 'Tor Browser couldn't connect to Tor' (alogn with the COnfigure Connection... button) fixes this)

Currently I have 'Try a Bridge' wired up to the AutoBootstrapping logic (which connects to rdsys via Moat to get settings) when in Bob's case they just need to disable their proxy and Connect again. Attempting to AutoBootstrap will (currently) fail if their are no settings for Bob's country and they would move on to the next screen asking for their location (to grab specific settings).

And speaking of that last problem:

@meskio, @cohosh: would it be ok for the bridge ecosystem to push users onto obfs4 bridges in the event that they are not in one of our special countries we have settings for?

Yes, I do think we should make easy to use bridges on those situations.

In Spain tor is not blocked per se, but my local university here does block it and you need a bridge (the default bridges work) to be able to connect. I think this situation is not rare, many corporate networks buy firewalls that come with options to block tor and the clueless sysadmin will enable everything thinking that they will be more secure. Will be great if we make it easy to provide anti-censorship tools for those situations.

first bootstrap fails because of proxy error

Wouldn't this get picked up by the initial "Online?" diamond in the user flows (which results in a "Check your internet connection" error with a Try again button) instead of redirecting the user to Connection Assist?

I'm not sure how/if we can differentiate bootstrap failure due to censorship from no internet.

I think we were going to base it on a successful Moat connection? The diamond was added in response to a similar issue that was brought up during a group review of a previous iteration of user flows.

From my notes in #29590 (comment 2749371):

Prior to bootstrapping, we could/should test the user's internet connection via moat in order to disambiguate moat being censored from internet connectivity issues.

Hmmm, so the problem with using the Moat connection (in this particular example) is that the Moat module uses the user's set proxy settings. So in Bob's case here the internet connectivity test would fail (granted I suspect Bob's use case isn't going to affect many users).

I think I need to go think a bit more on how the proxy and firewall settings should interact with censorship-circumvention.

So in Bob's case here the internet connectivity test would fail

Sorry, maybe I'm misunderstanding – but isn't that the result we want here?

Ah yes, of course.

So if I understand correctly:

If vanilla bootstrap fails, we then ping the Moat API. If that fails to connect, then we know there is an internet problem (either due to incorrect proxy/firewall settings, or the internet just being down), in which case we do not forward on to the censorship circumvention, but rather back to the configure page with appropriate copy indicating network connectivity problems.

Yep, that would work!

Although: are we already pinging Moat for circumvention settings/bridge updates before bootstrapping, and if so, would it make sense for that to be the basic connectivity test too?

So we are not pinging automatically for a couple of reasons (@meskio please let me know if these concerns are reasonable):

I don't want to knock over rdsys with requests when we turn this feature on for the entire tor browser user-base.
I also don't want to push through any traffic through the domain-fronted meek bridge than is necessary as it's my understanding that we has to pay for that traffic.

Ah gotcha. This will also affect the designs in #40782 (closed) then too, since I was betting on having pinged Moat on launch to inform the Internet connection state.

@richard your two concerns are kind of right:

I don't want to knock over rdsys with requests when we turn this feature on for the entire tor browser user-base.

I would expect rdsys to be able to handle the ping of the entire tor browser user base (as long as is just pinging to an API that is not providing bridges or any serious computation). But is better to don't create ourselves a bottleneck that we don't need here. If we end up needing it I will do some stress testing before we do this kind of change.

I also don't want to push through any traffic through the domain-fronted meek bridge than is necessary as it's my understanding that we has to pay for that traffic.

This is true, we do pay (currently fastly) for the traffic. I have no idea how expensive it is, we can look into it if needed.

Anyway, +1 to pinging moat if to connection has failed and not in all connections.

@meskio: hmm what if to test the internet connection we just make an XmlHttpRequest to the meek-azure url ( https://meek.azureedge.net/ )? This is where the connection goes through anyway right?

This is a good idea, you can do that. But AFAIK moat is not using meek azure's, is in fastly. The current domain name we are using is cdn.sstatic.net, but we might need to change it as is getting blocked in some countries (tpo/anti-censorship/pluggable-transports/snowflake#40068).

Anyway, using the same domain we are currently using for moat will be great. I guess you have it somewhere in the code and can just use the variable/config in both places. So when we decide to change the domain we don't have to change it in two places.

Another added value of not calling moat before every tor connection is to don't give information to fastly (or whatever CDN we use) of the IP address of every tor user.

@meskio: sorry, I don't understand how moat works apparently; is the entry point into rdsys defined by the bridge-line, or is it baked into the obfs4proxy executable itself?

Another added value of not calling moat before every tor connection is to don't give information to fastly (or whatever CDN we use) of the IP address of every tor user.

My thinking is to only do this query if bootstrapping with default setting fails to determine if we should direct the user to censorship circumvention UX or to general tor settings UX.

@meskio: sorry, I don't understand how moat works apparently; is the entry point into rdsys defined by the bridge-line, or is it baked into the obfs4proxy executable itself?

I'm not sure I understand totally how moat works, AFAIK you don't use a bridgeline but call directly meek-client (or is it obfs4proxy?). I thought it was not using obfs4proxy, but meek-client, but I just found out yesterday that obfs4proxy does also implement the meek protocol. Maybe @cohosh knows.

My thinking is to only do this query if bootstrapping with default setting fails to determine if we should direct the user to censorship circumvention UX or to general tor settings UX.

In that case I'm ok with TB calling to moat. At this point you can call any API endpoint like /countries, /map or /builtin will not require any computation and be pretty fast. I see you were asking for a /ping API, any of the others (except /settings) will be as fast as whatever I could do with /ping, but if it will make your life easier I will add it.

Hi!

I am trying to implement the check to detect if user can access the Internet.

So, @meskio do you confirm me I can use one of the pages you already listed (like /countries)? Or in case, could you please provide me some other endpoint?

The test would not be run by each Tor user, but only when you click on a new button I am adding for this purpose. And I am modifying the code to perform a HEAD request, rather than a POST, since we really do not need any content for this test request.

So, @meskio do you confirm me I can use one of the pages you already listed (like /countries)? Or in case, could you please provide me some other endpoint?

Yes, if tor has failed to connect I think /countries is a good endpoint to call.

And I am modifying the code to perform a HEAD request, rather than a POST, since we really do not need any content for this test request.

I'm not sure how the HEAD request work and if my current code will respond to it. But if it doesn't let me know and I'll fix it.

Thanks! I have already implemented it with HEAD, and it works as expected

According to a test performed on November 2021, users are not reading the information about Tor Browser using their location when they click on try a bridge (see tpo/ux/research#52 (closed) for full report). On this test, one out of 5 users read the information about allowing Tor Browser to use their location, and highlighted that they were not sure if this would compromise their privacy and security. They also said they tend to follow the purple button to connect successfully.

We could try other options with users to make sure they understand that Tor Browser is using their location to find an useful bridge.

Some options:

new tb-manual to inform Tor's need to access location
highlight the information about using their location
new modal on Try a bridge to explicitly ask consent to access users' location

mentioned in issue #40774 (closed)

mentioned in issue tpo/anti-censorship/bridgedb#40043 (closed)

mentioned in issue #40796 (moved)

mentioned in issue tpo/ux/design#39 (closed)

marked this issue as related to tpo/ux/design#39 (closed)

marked this issue as related to #40544 (closed)

mentioned in issue tpo/anti-censorship/team#69 (closed)

mentioned in issue tpo/ux/research#71 (closed)

added Q2 label

mentioned in issue #40874 (closed)

mentioned in issue #40875 (closed)

I've iterated on the design for Connection Assist a little further following some design QA of @pierov's latest work in #40773 (closed), and in preparation for the next round of usability testing happening in Brazil & Mexico in tpo/ux/research#71 (closed).

@nicob has also been working on an improved set of Connection icons (among others) here: tpo/ux/design#39 (closed). The final icons haven't been selected yet and are currently being tested, so I'll post a followup ticket when we're ready to add these in.

Internet error

In the user flows (about-torconnect-user-flows.png) and following the conversation above (#40781 (comment 2772219)), we decided that users who fail to connect and can't reach moat either should get redirected to an Internet connection error instead.

I think it would be useful to explain that the user should check for network/proxy issues and try again, before attempting to config a bridge? I could use some help with the wording though please.

Connection error

The most notable changes here affect the breadrumb and actions:

Breadcrumb states

I've made some improvements to the breadcrumb tabs' colors, hover and pressed states to bring them more inline with the about:preferences tabs (albeit on a smaller scale), and have pushed these to the Proton library: Figma link

(Hidden among those changes is also a shift from blue-60 to blue-50, which is what Firefox uses in this context.)

I think these look and work a little better in practice – see this screen here, for example: Figma prototype

Breadcrumb structure

Although they number the same as before, I've shuffled around the tabs a little too. "Connect to Tor" is now present as the first item in the breadcrumb to facilitate backwards-navigation, and allow the user to reattempt a regular connection – addressing the wider issue described in #40874 (closed).

Following user feedback from the usability testing @nah conducted in tpo/ux/research#52 (closed), where only one user realized Tor Browser was using their location when attempting to try a bridge, I've combined the Connection Assist and Location Settings tabs into one to bring the location UI forward and ensure we get informed consent for this feature.

Given that change, we'll also need to incorporate and default to the "Automatic" option that's currently present in Connection Settings (see #40782 (comment 2771120) for reference).

However I would hold fire on combing the Connection Assist and Location Settings tabs until the current round of usability testing is complete, as I may need to change approach again depending on the results.

Actions

In order to streamline the UX I think it would be beneficial to limit the buttons to two actions (including the selector). To do so I've moved "Configure Connection..." into the description as an inline link where necessary – with two notable exceptions: torconnect's initial Connect to Tor screen, and the final error where it's presented as the recommended option.

Again, we may want to wait for the outcome of Nah's usability testing before making this change in dev.

To help limit the overall vertical height of the content, I've wrapped also the "View logs..." link onto the same line as the error string too. This may be applied immediately.

Connect to Tor

Quick example demonstrating the backwards navigation in action. However, Connect to Tor should remain breadcrumbless by default as per the designs in #40780 (closed), unless Connection Assist has been fired up in response to a connection error.

Note that the Connect button now reads "Try Again".

Autolocate error

I don't think we currently include an error for when autolocate fails – maybe we were just forwarding them onto the next tab to add their location manually? However if we combine these tabs, we'll need a dedicated error screen.

Trying a bridge

Location check

We're currently testing moving the red error highlighting from the tab to the actions themselves, since this seems like a more useful place to draw the user's attention.

Last try

Final error

All of the above UI has been moved to a new page in Figma: Figma link

And there's a new prototype that demos a moderate censorship scenario available here: Figma prototype

We're currently testing moving the red error highlighting from the tab to the actions themselves, since this seems like a more useful place to draw the user's attention.

What about having both red?

I actually had it that way originally, but @richard pointed out it was a little much and in retrospect I probably agree.

marked this issue as related to #40863

Re: automatic Internet Test

Bootstrap does not exceed either 2% (with bridges) or 5% (without) when I test it with my Ethernet cable unplugged.

I think that we could therefore cancel the automatic Internet test when we exceed 10%, to avoid additional fingerprinting and load on Moat. But we should ask the opinion of the anticensorship team for the best timeout.

At the moment I set the Internet test either at 30s +- 5s from the first bootstrap, and usually the bootstrap is quicker than that (with "normal" connections). I don't know with problematic connections how long it usually takes.

Do you have any data from the UR?

The +- 5s are to make fingerprinting a little bit harder, even though probably a censor could detect Tor/Tor Browser anyway.

Sometimes, the bootstrap also fails before the 30s (usually with an error like "no route to host - IP address"). Currently my ideas are:

see if tor already decided that it means that the Internet is down - which I expect it does, because it thinks it's offline by default;
anticipate the Internet test, and delay the UI to when we have the result of the Internet test.

Please notice that the Internet test takes a little while to declare the offline status (basically, the connection has to go to timeout - which takes 20-30s).

Finally, the Internet test now communicates the date of the Moat server, as we commented in #40863.

In the current implementation, I removed the check on the bootstrap progress, but the rest is implemented as I wrote here .

From my perspective, this kind of connectivity check does give censors some new information, but this will not help them to the point that gives them an advantage. So I think this shouldn't be an issue.

If we want to absolutely be on the safe side, we can request the user to click on a button before proceeding with additional diagnosing.

I wonder if there is anything else to add from @meskio, @cohosh.

What you shay @shelikhoo makes sense.

I think 30s should be a good timeout to starts with. We can always tune it up if needed, but anyway waiting for more than 30s might mean users having a pretty bad experience using tor.

mentioned in issue fenix#40214 (moved)

marked this issue as related to fenix#40214 (moved)

marked this issue as related to #29590 (closed)

added Sponsor 96 label

mentioned in issue #40888 (closed)

mentioned in issue #40891 (closed)

At the moment, we're showing the «Tor Browser couldn't locate you» whenever we don't receive any settings.

It's a bad error: we could probably locate the user, so we should just ask confirmation on their location. So, we could ask the anti-censorship team to provide us the detected location even though we don't have settings for it.

However, I recognize this may be scaring: we're promising to make users anonymous, but then show them their location.

Another problem is that we could try the default settings (tpo/anti-censorship/bridgedb#40045 (closed)), but we aren't doing it.

I think we could try to do so after each automatic bootstrap, if circumvention/settings fails (either with or without a country specified by the user).

However, I recognize this may be scaring: we're promising to make users anonymous, but then show them their location.

This is interesting. In the recent Usability Testing with users from Brazil and Mexico, and the majority don't understand why Tor needs their location (even, if it says 'to find the right bridge'). So, they assume Tor wants to give them a bridge that is closer to them, so instead of choosing their own country in the list of countries, they choose one that is geographically near to their own.

The other common issue is 'isn't it danger to share my location?', these users tend to configure manually their bridge instead of choosing the automatic option.

I'm still on my way to analyze the results, and have some usability testing to do this week and in the next one, but gonna share here as soon as I have new findings.

Great, thanks!

Here's a quick summary of a convo we had on IRC about this topic:

When the user opts-in to autolocation we send a request to Moat
Moat doesn't tell us where the browser's connecting from, it just provides settings (if there are settings available for the user's location) or no settings (if it doesn't)
Otherwise, if we can't reach Moat the user will get the Internet connection error instead

So, we could:

Apply the fallback settings in the scenario we get no settings
Include the country in Moat's response, and display the location-check error should bootstrapping fail again*
Display the autolocate error only if Moat doesn't return any country at all (which will presumably be a very rare occurrence)

Is that correct @pierov?

*I don't think this will be an issue from a UX point of view since the user has already opted-in to autolocation

Yes, correct!

We could try the fallback settings if we don't get settings from the autolocate, and ask for a location confirmation only if they fail as well.

Great, that sounds good to me!

Should we wait for a meeting with the anti-censorship team, or should I just proceed and implement this change?

To get the location we will need their help, because it's on the Moat code.

I think we can proceed with:

Trying the fallback settings when the browser receives no settings from Moat
Forwarding the user to the Internet connection error when the browser can't reach Moat

But otherwise wait for @meskio's opinion on the rest?

I think this decision makes sense, using the moat provided 'default' settings if there is none for the country we located should be a good fallback.

I can provide the detected country location as an extra field in the /moat/circumvention/settings json or another API endpoint if that will make your life easier.

The extra field would be great, thank you!

I just created an issue in rdsys to work on it: tpo/anti-censorship/rdsys#108 (closed)

I'll try to do it soonish.

removed Doing label

added Next label

closed

mentioned in issue #41188 (closed)

Design new flow to circumvent censorship

Designs

Child items ...

Activity