gather info on jump-to-80% issues
This ticket is now for gathering additional info and feedback about the category of jump-to-80% bootstrap reporting problems.
background:
If enough existing directory information is available, the bootstrap progress as reported to the logs and the control connection jumps from 0% to 80% almost immediately. This is misleading and causes user frustration, as reported by Linda's study.
When bootstrapping with existing directory information, we should rescale the progress numbers so they advance on something resembling a linear time scale, which is probably closer to what users expect to see.
Activity
changed milestone to %Tor: 0.4.0.x-final
Replying to catalyst:
If enough existing directory information is available, the bootstrap progress as reported to the logs and the control connection jumps from 0% to 80% almost immediately. This is misleading and causes user frustration, as reported by Linda's study.
I was one of the people who helped with the study. As I understand it, the problem has more to do with Tor Launcher than with tor. I don't think the problem has to do with cached directory information. Rather, it is that Tor Launcher resets the state of the progress bar to 0% every time the progress screen is displayed, even though the tor process's (hidden) percentage is greater than 0%.
Imagine you start bootstrapping and watch the progress bar get to 60%, then you get impatient and hit Cancel. (tor is in fact still bootstrapping in the background even though the GUI doesn't reflect that.) You fiddle with the configuration and try to bootstrap again. At this point, the progress bar misleadingly shows 0%, even though the tor process is still at 60%. Now, as soon as tor makes a little more progress (say to 65%), the progress bar will immediately update itself to the new value, giving the effect of a jump from 0% to 65% when it should have been from 60% to 65%. What users found misleading was the progress bar going back to 0% after they changed the configuration; even though in reality the percentage hadn't changed or had even increased, they assumed that their configuration changes had caused bootstrapping to make negative progress.
I thought that the solution would be to have Tor Launcher either cache its last seen progress percentage, so it can reinitialize the progress bar properly, or else have some background listener that tracks the percentage status even when the progress bar is not actually on screen.
I might be misinterpreting the ticket description.
When bootstrapping with existing directory information, we should rescale the progress numbers so they advance on something resembling a linear time scale, which is probably closer to what users expect to see.
Does that mean that if bootstrapping got to 60%, was cancelled, and then restarted, that the progress bar would visually reset back to 0%, but that the remaining 40% would be stretched to fit a 0–100 scale? E.g. 60%→0%, 70%→25%, 80%→50%, 90%→75%, 100%→100%?
Replying to dcf:
I was one of the people who helped with the study. As I understand it, the problem has more to do with Tor Launcher than with tor. I don't think the problem has to do with cached directory information. Rather, it is that Tor Launcher resets the state of the progress bar to 0% every time the progress screen is displayed, even though the tor process's (hidden) percentage is greater than 0%.
Indeed, this is what Tor Launcher does. When we wrote the code, we assumed that progress would occur reasonably quickly when things were going to work (which means that tor would send another bootstrap status message to Tor Launcher soon, so starting at zero would not matter much). If instead there is a long wait before the next bootstrap status message, things will not look good to the user.
Imagine you start bootstrapping and watch the progress bar get to 60%, then you get impatient and hit Cancel. (tor is in fact still bootstrapping in the background even though the GUI doesn't reflect that.)
We tried to address that in #11879 (moved) by changing Tor Launcher to do
SETCONF DisableNetwork=1when the user cancels. But note related tickets #15713 (moved) and #15715 (moved).You fiddle with the configuration and try to bootstrap again. At this point, the progress bar misleadingly shows 0%, even though the tor process is still at 60%. Now, as soon as tor makes a little more progress (say to 65%), the progress bar will immediately update itself to the new value, giving the effect of a jump from 0% to 65% when it should have been from 60% to 65%. What users found misleading was the progress bar going back to 0% after they changed the configuration; even though in reality the percentage hadn't changed or had even increased, they assumed that their configuration changes had caused bootstrapping to make negative progress.
Based on your description, it seems like this behavior violates most people's "mental model" of what is happening underneath the covers. I think it would be okay to reset the progress to zero if progress was then made at a rate similar to when the previous configuration settings were used (to the user, this is an "everything starts from scratch" model). Or picking up where things left off, e.g., at 60%, would also be okay with most users, although if there is a lengthy stall there they may still give up. Finer-grained progress reporting would help.
I thought that the solution would be to have Tor Launcher either cache its last seen progress percentage, so it can reinitialize the progress bar properly, or else have some background listener that tracks the percentage status even when the progress bar is not actually on screen.
Here is a question for the Network Team: would it be safe for Tor Launcher to cache the progress value? Will things ever move backward? I don't think adding caching to Tor Launcher would be difficult at all, and Tor Launcher is monitoring the bootstrap status events even when its progress window is not open so in theory it could always know the current progress value.
When bootstrapping with existing directory information, we should rescale the progress numbers so they advance on something resembling a linear time scale, which is probably closer to what users expect to see.
Does that mean that if bootstrapping got to 60%, was cancelled, and then restarted, that the progress bar would visually reset back to 0%, but that the remaining 40% would be stretched to fit a 0–100 scale? E.g. 60%→0%, 70%→25%, 80%→50%, 90%→75%, 100%→100%?
I have to say that I don't much like they idea of "rescaling" the progress. I think it would be better to promote a model to the user that lets them know that half way across means the same thing every time, in every copy of Tor Browser, whether it is their own browser or that of a friend they are trying to help.
Replying to mcs:
Replying to dcf:
I was one of the people who helped with the study. As I understand it, the problem has more to do with Tor Launcher than with tor. I don't think the problem has to do with cached directory information. Rather, it is that Tor Launcher resets the state of the progress bar to 0% every time the progress screen is displayed, even though the tor process's (hidden) percentage is greater than 0%.
Indeed, this is what Tor Launcher does.
It seems like rather than resetting to 0%, tor launcher should ask tor what its current bootstrap progress is, and then use that for starters. That is, the simple bug here is that Tor Launcher puts the progress bar back to 0, and then waits for Tor to tell it a new progress number, yet Tor will only emit an event when it's made further progress beyond what it told you about in the last event.
So yes, Tor Launcher should either remember the current level of progress, or if it wants to be stateless (maybe that's more elegant?), do a getinfo to fetch it whenever you're going to reset the progress bar.
Based on your description, it seems like this behavior violates most people's "mental model" of what is happening underneath the covers. I think it would be okay to reset the progress to zero if progress was then made at a rate similar to when the previous configuration settings were used
Agreed, but then you need to update it to the amount of progress that Tor thinks it has told you it's made. Tor doesn't know that you reset the progress bar, so it can't know that it needs to send you another (redundant, from its perspective) bootstrap event.
Here is a question for the Network Team: would it be safe for Tor Launcher to cache the progress value? Will things ever move backward? I don't think adding caching to Tor Launcher would be difficult at all, and Tor Launcher is monitoring the bootstrap status events even when its progress window is not open so in theory it could always know the current progress value.
That's a great question. I will let the network team decide on their future answer, but as for what the code does right now, bootstrap_percent never moves backwards, and Tor tries to avoid emitting any bootstrap progress events unless the new number is bigger than the number from the last event.
(Should Tor Launcher be doing the getinfo anyway, on startup, in case it missed a few bootstrap events before it connected to the control port?)
I have to say that I don't much like they idea of "rescaling" the progress. I think it would be better to promote a model to the user that lets them know that half way across means the same thing every time, in every copy of Tor Browser, whether it is their own browser or that of a friend they are trying to help.
I agree. (But I am open to being overruled by Linda et al if they can help us do better. :)
-
We are working on improving things on the Tor Launcher side. I opened #23240 (moved).
-
Although the Tor Launcher fix in #23240 (moved) has been merged and seems to work, as Kathy and I are working on the revised Tor Launcher UI we have discovered that the approach used in our fix for #23240 (moved) no longer works. Specifically, after implementing #23262 (moved) (integrated progress bar), we see that a tor that has cached directory information (80% start) returns
PROGRESS=0in response to theGETINFO status/bootstrap-phasecommand that Tor Launcher sends. This is surprising; Kathy and I think the right answer isPROGRESS=80. The problem with receiving zero is that our strategy for #23240 (moved) was to keep the progress bar hidden until we queried tor to find out the current bootstrap progress value.Tor Launcher starts tor with DisableNetwork=1 on the command line, and here is the sequence of control port commands that is issued:
SETCONF UseBridges Bridge SETCONF Socks4Proxy Socks5Proxy Socks5ProxyUsername Socks5ProxyPassword HTTPSProxy HTTPSProxyAuthenticator SETCONF ReachableAddresses SETCONF DisableNetwork=0 SAVECONF GETINFO status/bootstrap-phaseIt is that last command that returns
PROGRESS=0.Can someone on the networking team confirm that this is occurring due to the internal architecture of tor? E.g., when
DisableNetwork=0is received when networking is disabled, the process of enabling the networking code must be asynchronous (which leads to the confusingGETINFO status/bootstrap-phaseresponse). That would explain problems like #15715 (moved) as well (and we do see what to us are spurious "Disable Network is set" NOTICE message in our testing because the commands above are immediately followed by someGETCONFcommands).With Tor Launcher's current, separate progress window things work okay because there is enough delay between the
SETCONF DisableNetwork=0and theGETINFO status/bootstrap-phasecommands that tor returnsPROGRESS=80.Trac:
Status: new to needs_information The controller-visible bootstrap phase always starts from 0 when tor starts up (even if DisableNetwork is 1, I think); phase 0 is always the first bootstrap event sent to the control connection. The trigger for reaching phase 80 is having a consensus (though not necessarily an up-to-date-one; this is probably a bug!) and enough descriptors to build circuits. This might not happen immediately upon setting DisableNetwork=0. I haven't checked all of the ways this happens -- the relevant bootstrap event reporting is in
update_router_have_minimum_dir_info()which is called byrouter_have_minimum_dir_info(), and the latter gets called through several paths, some of which look like timer-triggered events that occur once per second.That said, I think there are several reasons to not show a bootstrap phase of 80 in circumstances where we currently do so:
- The consensus might be stale, so we should download a new one.
- We might not be able to connect to any relays. This means we might not get anywhere near 80 in our current configuration, maybe not even as far as 10 or 20.
- Our clock might be skewed by several hours, so we might be stuck at 80 forever trying to download an up-to-date consensus (with very large exponential timeouts) and never succeeding. (See #23508 (moved).)
-
Replying to catalyst:
The controller-visible bootstrap phase always starts from 0 when tor starts up (even if DisableNetwork is 1, I think); phase 0 is always the first bootstrap event sent to the control connection. The trigger for reaching phase 80 is having a consensus (though not necessarily an up-to-date-one; this is probably a bug!) and enough descriptors to build circuits. This might not happen immediately upon setting DisableNetwork=0....
Thanks for the detailed explanation. Given what you said, I think it is fine for Tor Launcher to show a progress bar at position zero when that is tor's internal state. The scenario that bothered people in the Linda/Berkeley study is still addressed by the fix in #23240 (moved) (wherein Tor Launcher retrieves the current status before showing the progress bar).
Trac:
Status: needs_information to new
