The Tor Project issueshttps://gitlab.torproject.org/groups/tpo/-/issues2020-06-27T13:50:38Zhttps://gitlab.torproject.org/tpo/core/tor/-/issues/29813Add unmeasured and vote Line KeyValues in the bandwidth-file-spec2020-06-27T13:50:38ZjugaAdd unmeasured and vote Line KeyValues in the bandwidth-file-specAs commented in https://trac.torproject.org/projects/tor/ticket/28563#comment:23As commented in https://trac.torproject.org/projects/tor/ticket/28563#comment:23Tor: 0.4.1.x-finaljugajugahttps://gitlab.torproject.org/tpo/core/tor/-/issues/29806Ignore bandwidth file lines with "vote=0"2020-06-27T13:50:38ZjugaIgnore bandwidth file lines with "vote=0"As commented in https://trac.torproject.org/projects/tor/ticket/28563#comment:14As commented in https://trac.torproject.org/projects/tor/ticket/28563#comment:14Tor: 0.3.4.x-finalteorteorhttps://gitlab.torproject.org/tpo/core/tor/-/issues/2980541 coverity defects on prob_distr.c2020-06-27T13:50:38ZGeorge Kadianakis41 coverity defects on prob_distr.cThe `DIST_BASE_TYPED` macro in `prob_distr.h` is causing us 41 new coverity defects. I don't think it's wrong but it's quite hacky so we should fix it in some way.
```
*** CID 1444029: Incorrect expression (SIZEOF_MISMATCH)
/src/lib/m...The `DIST_BASE_TYPED` macro in `prob_distr.h` is causing us 41 new coverity defects. I don't think it's wrong but it's quite hacky so we should fix it in some way.
```
*** CID 1444029: Incorrect expression (SIZEOF_MISMATCH)
/src/lib/math/prob_distr.c: 1511 in log_logistic_sf()
1505 return cdf_log_logistic(x, LL->alpha, LL->beta);
1506 }
1507
1508 static double
1509 log_logistic_sf(const struct dist *dist, double x)
1510 {
>>> CID 1444029: Incorrect expression (SIZEOF_MISMATCH)
>>> Adding "0UL /* 0 * sizeof (dist - &((struct log_logistic const *)((char const *)dist - __builtin_offsetof()))->base) */" to pointer
>>> "(struct log_logistic const *)((char const *)dist - 0UL)" of type "struct log_logistic const *" is suspicious because adding an integral value
>>> to this pointer automatically scales that value by the size, 24 bytes, of the pointed-to type, "struct log_logistic const". Most likely, the
>>> multiplication by "sizeof (dist - &((struct log_logistic const *)((char const *)dist - 0UL))->base)" in this expression is extraneous and
>>> should be eliminated.
1511 const struct log_logistic *LL = const_container_of(dist,
1512 struct log_logistic, base);
1513
1514 return sf_log_logistic(x, LL->alpha, LL->beta);
1515 }
1516
```Tor: 0.4.1.x-finalhttps://gitlab.torproject.org/tpo/core/tor/-/issues/29802Document the v3 onion service key files in the tor man page2022-09-01T21:29:27ZteorDocument the v3 onion service key files in the tor man pageThe tor man page is missing the names of the key files for v3 onion services.The tor man page is missing the names of the key files for v3 onion services.https://gitlab.torproject.org/tpo/core/tor/-/issues/29801Write a proposal for IPv6 "Happy Eyeballs" on Tor clients2020-06-27T13:50:38ZNeel Chauhanneel@neelc.orgWrite a proposal for IPv6 "Happy Eyeballs" on Tor clientsTeor has asked for updates in https://github.com/torproject/torspec/pull/61, but this PR has already been merged.
I have created a new PR for his suggestions: https://github.com/torproject/torspec/pull/63
I don't believe you can add to...Teor has asked for updates in https://github.com/torproject/torspec/pull/61, but this PR has already been merged.
I have created a new PR for his suggestions: https://github.com/torproject/torspec/pull/63
I don't believe you can add to a merged PR so that's why a new PR exists.Tor: 0.4.3.x-finalteorteorhttps://gitlab.torproject.org/tpo/core/tor/-/issues/29793show-distdir-core and friends should fail more quietly2022-06-17T16:35:49ZTaylor Yushow-distdir-core and friends should fail more quietly`make show-distdir-core` and `make show-distdir-testlog` should probably fail more quietly so they don't produce spurious errors from `make`. These spurious errors can make troubleshooting actual build problems more difficult.
(Yes, Tra...`make show-distdir-core` and `make show-distdir-testlog` should probably fail more quietly so they don't produce spurious errors from `make`. These spurious errors can make troubleshooting actual build problems more difficult.
(Yes, Travis ignores them, but `make` doesn't.)https://gitlab.torproject.org/tpo/core/tor/-/issues/29792practracker problems and CI broken on master2021-09-16T14:24:10ZGeorge Kadianakispractracker problems and CI broken on masterCI is broken on master because of the following practracker issues:
```
python3 ./scripts/maint/practracker/practracker.py .
problem function-size /src/feature/nodelist/nodelist.c:compute_frac_paths_available() 193
problem file-size /src...CI is broken on master because of the following practracker issues:
```
python3 ./scripts/maint/practracker/practracker.py .
problem function-size /src/feature/nodelist/nodelist.c:compute_frac_paths_available() 193
problem file-size /src/core/or/circuituse.c 3150
```
These were caused by legacy/trac#28656 and legacy/trac#29665.
They were not detected during the PR-check phase, because the PR was for 035 and 040 which dont include the practracker check. How should we avoid this issue in the future?Tor: 0.4.2.x-finalteorteorhttps://gitlab.torproject.org/tpo/core/tor/-/issues/29789practracker.py codec exception in some locales2020-06-27T13:50:39ZTaylor Yupractracker.py codec exception in some localespractracker.py, implemented in legacy/trac#29221, seems to have a locale dependency when python3 is being used. If the locale isn't a UTF-8 locale, UTF-8 characters in sources can result in an exception:
```
$ LANG=en_US.US-ASCII make ...practracker.py, implemented in legacy/trac#29221, seems to have a locale dependency when python3 is being used. If the locale isn't a UTF-8 locale, UTF-8 characters in sources can result in an exception:
```
$ LANG=en_US.US-ASCII make check-best-practices PYTHON=python
python ../scripts/maint/practracker/practracker.py ..
mirkwood:build-norust tlyu$ LANG=en_US.US-ASCII make check-best-practices
python3 ../scripts/maint/practracker/practracker.py ..
Traceback (most recent call last):
File "../scripts/maint/practracker/practracker.py", line 151, in <module>
main()
File "../scripts/maint/practracker/practracker.py", line 134, in main
found_new_issues = consider_all_metrics(files_list)
File "../scripts/maint/practracker/practracker.py", line 89, in consider_all_metrics
found_new_issues |= consider_metrics_for_file(fname, f)
File "../scripts/maint/practracker/practracker.py", line 104, in consider_metrics_for_file
found_new_issues |= consider_file_size(fname, f)
File "../scripts/maint/practracker/practracker.py", line 51, in consider_file_size
file_size = metrics.get_file_len(f)
File "/Users/tlyu/src/tor/scripts/maint/practracker/metrics.py", line 11, in get_file_len
for i, l in enumerate(f):
File "/Users/tlyu/src/brew/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/encodings/ascii.py", line 26, in decode
return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 14: ordinal not in range(128)
make: *** [check-best-practices] Error 1
```
I'm also seeing this on gitlab.com CI, but I don't know offhand what its locale environment variables are.
We might want to use the `encoding=` keyword parameter to `open()`, but I think that would no longer be python2 compatible.Tor: 0.4.1.x-finalTaylor YuTaylor Yuhttps://gitlab.torproject.org/tpo/core/tor/-/issues/29786Path bias circuits can still have cells pending2022-06-24T14:37:39ZMike PerryPath bias circuits can still have cells pendingIn https://gitlab.torproject.org/tpo/core/tor/-/issues/25573, we realized that half-closed connections need to be checked for extra cells when the circuit has been switched to path bias testing. The checks were added to the top of circui...In https://gitlab.torproject.org/tpo/core/tor/-/issues/25573, we realized that half-closed connections need to be checked for extra cells when the circuit has been switched to path bias testing. The checks were added to the top of circuit_receive_relay_cell(), by calling pathbias_check_probe_response() to check if the path bias probe was correct, and if not, we call pathbias_count_valid_cells() to check if the cell is from a previous half-closed connection.
In https://github.com/mikeperry-tor/vanguards/issues/37, we learned that path bias circuits can still have a pending cell for onion services. In particular, there can be outstanding cells for RELAY_COMMAND_INTRO_ESTABLISHED, RELAY_COMMAND_RENDEZVOUS_ESTABLISHED, and RELAY_COMMAND_INTRODUCE_ACK, depending on circuit type.
There's sloppy ways to fix this, which are easy (just hack pathbias_count_valid_cells() to allow 1 cell for those circuit types) and precise ways (actually track if the pending cell has been received or not before and after path bias transition).
We should probably fix this the precise way, and just implement the hacky workaround in vanguards for now.https://gitlab.torproject.org/tpo/core/tor/-/issues/29782Multiple SocksPort is broken, connects to entry node multiple times.2022-06-17T18:25:05ZcypherpunksMultiple SocksPort is broken, connects to entry node multiple times.What the fuck is going on here?
If I use multiple SocksPort, it connects to entry node multiple times, instead of one time.
So CIA and NSA can analyze my traffic more easily. They also know how many applications I use with Tor.
That's hu...What the fuck is going on here?
If I use multiple SocksPort, it connects to entry node multiple times, instead of one time.
So CIA and NSA can analyze my traffic more easily. They also know how many applications I use with Tor.
That's huge bug. There should be one connection to entry node, but then each socksport should use different middle and exit node. (or maybe use same middle node too?)
You are just helping NSA. Do they own torproject?
Here is how to get the bug:
1. Configure Tor to use multiple SocksPort with IsolateDestAddr flag
2. Start Tor
3. Connect each application to each SocksPort and start doing network activity on all of them.
4. You might get multiple TCP connections to entry node.
5. Each separate TCP connection transmits data from separate SocksPort.
It doesn't happen 100% of time. Sometimes you need to wait or try again to get this bug.
This bug is a design flaw maybe. It lowers privacy and gives zero benefits.
NSA, CIA, can isolate each TCP connection and try to make analysis and correlation. If everything was transmitted on single TCP connection they would need to own entry node to do same thing. If everything was transmitted on single Entry and Middle node (but different Exit node) they would need to own entry and middle node to make this analysis.https://gitlab.torproject.org/tpo/core/tor/-/issues/29780Run travis with python2 and python32020-07-29T03:19:41ZteorRun travis with python2 and python3We should make sure all of tor's test python scripts run in environments where python3 is the default.
(Scripts are also allowed to explicitly require python3. But they can't depend on python2, because it is end of life on 1 Jan 2020.)
...We should make sure all of tor's test python scripts run in environments where python3 is the default.
(Scripts are also allowed to explicitly require python3. But they can't depend on python2, because it is end of life on 1 Jan 2020.)
We should be able to copy the python bits of our chutney travis config into Tor.
Let's just pick the newest python 3?Tor: unspecifiedhttps://gitlab.torproject.org/tpo/core/tor/-/issues/29777Rate-limit "Problem bootstrapping" warnings to one every 5 seconds2022-02-07T19:39:00ZteorRate-limit "Problem bootstrapping" warnings to one every 5 secondsLet's put a rate-limit on warnings like this:
```
Jan 29 11:36:27.000 [warn] Problem bootstrapping. Stuck at 0% (starting): Starting. (Network is unreachable; NOROUTE; count 11; recommendation warn; host 322C6E3A973BC10FC36DE3037AD27BC89...Let's put a rate-limit on warnings like this:
```
Jan 29 11:36:27.000 [warn] Problem bootstrapping. Stuck at 0% (starting): Starting. (Network is unreachable; NOROUTE; count 11; recommendation warn; host 322C6E3A973BC10FC36DE3037AD27BC89F14723B at 212.83.154.33:8443)
```https://gitlab.torproject.org/tpo/core/tor/-/issues/29776Mark proposal 301 as "Accepted", and update the index2020-06-27T13:50:40ZteorMark proposal 301 as "Accepted", and update the indexTor: 0.4.1.x-finalteorteorhttps://gitlab.torproject.org/tpo/core/tor/-/issues/29767Prob distr stochastic tests fail CI (innocuous?)2020-06-27T13:50:40ZGeorge KadianakisProb distr stochastic tests fail CI (innocuous?)The stochastic tests are failing the CI as we expected. This is a ticket to collect such failures, to eventually see if they are really innocuous or there is some underlying bug. At this point I have two such failures collected both for ...The stochastic tests are failing the CI as we expected. This is a ticket to collect such failures, to eventually see if they are really innocuous or there is some underlying bug. At this point I have two such failures collected both for the same stochastic test (log-logistic):
```
slow/prob_distr/stochastic_log_logistic: [forking] fail log logistic sampler
another stochastic test fail
Seed: D3CF5E3899188E3357D47061E936FF96
```
```
slow/prob_distr/stochastic_log_logistic: [forking] fail log logistic sampler
FAIL ../src/test/test_prob_distr.c:1390: assert(ok)
NOTE: This is a stochastic test, and we expect it to fail from
time to time, with some low probability. If you see it fail more
than one trial in 100, though, please tell us.
Seed: 883CEA3493C96F8D59EE7A6554EB594E
```Tor: unspecifiedhttps://gitlab.torproject.org/tpo/core/tor/-/issues/29756Check for correct header macro guards.2020-06-27T13:50:40ZNick MathewsonCheck for correct header macro guards.We should probably make sure that our headers all have correct guard macros. When we forget to do this, we usually get sad about it.We should probably make sure that our headers all have correct guard macros. When we forget to do this, we usually get sad about it.Tor: 0.4.1.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/29754Include new monitoring KeyValues in the bandwidth-file-spec2020-06-27T13:50:40ZjugaInclude new monitoring KeyValues in the bandwidth-file-specAfter legacy/trac#28547 is implemented, there will be new KeyValues that need to be documented in the bandwidth file spec.After legacy/trac#28547 is implemented, there will be new KeyValues that need to be documented in the bandwidth file spec.Tor: 0.4.1.x-finalteorteorhttps://gitlab.torproject.org/tpo/core/tor/-/issues/29747shellcheck errors on the new git scripts2020-06-27T13:50:40ZGeorge Kadianakisshellcheck errors on the new git scriptsThe scripts added by legacy/trac#29391 are triggering shellcheck errors on travis:
https://travis-ci.org/asn-d6/tor/jobs/505198755#L3149
CI seems to be partially broken because of that.The scripts added by legacy/trac#29391 are triggering shellcheck errors on travis:
https://travis-ci.org/asn-d6/tor/jobs/505198755#L3149
CI seems to be partially broken because of that.Tor: 0.4.1.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/29746Improve Tor best practices tracker2021-09-16T14:24:10ZGeorge KadianakisImprove Tor best practices trackerVarious improvements could be done to the best-practices tracker (practracker legacy/trac#29221). Here are a few from Nick's last review:
https://github.com/torproject/tor/pull/742#discussion_r262564420
https://github.com/torproject/tor...Various improvements could be done to the best-practices tracker (practracker legacy/trac#29221). Here are a few from Nick's last review:
https://github.com/torproject/tor/pull/742#discussion_r262564420
https://github.com/torproject/tor/pull/742#discussion_r262565153
~~https://github.com/torproject/tor/pull/742#discussion_r262566041~~
https://github.com/torproject/tor/pull/742#discussion_r262566501
https://github.com/torproject/tor/pull/742#discussion_r262567882
This could also be a fine starting volunteer project.Tor: 0.4.2.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/29744Streams sometimes stall for up to 1 hour without making any progress2022-09-01T21:29:27ZKarsten LoesingStreams sometimes stall for up to 1 hour without making any progressWe're measuring Tor performance using our OnionPerf tool by regularly downloading 5 MiB files over Tor. Some of these measurements run longer than 1 hour, after which a timeout in OnionPerf aborts them, or run for up to 30 minutes until ...We're measuring Tor performance using our OnionPerf tool by regularly downloading 5 MiB files over Tor. Some of these measurements run longer than 1 hour, after which a timeout in OnionPerf aborts them, or run for up to 30 minutes until they complete. (For comparison, 99% of successful runs complete within roughly two minutes.)
I noticed one particular source of slowness which I think is the reason for the application timeouts after 1 hour and for some of the 1% slowest successful runs: streams stall for seconds or minutes and would even stall for hours if we let them, without making any progress; and suddenly they make progress until they complete or stall again.
I'm attaching four graphs showing this problem. All these graphs show download progress over time with time on x and progress on y. Each gray bar is one measurement. The black line starts at the bottom of its gray bar and goes up to the top of that bar as more data is received. The number on the right is the stream ID.
The first two graphs show application timeouts, the last two show the slowest 1% of successful runs. First and third show downloads from a public server, second and fourth from an onion server.
Note that not all runs have this problem of stalling as described above. Some of the more obvious cases are:
- Page 3, stream ID 436971: that stream basically does nothing for over half an hour and then completes within seconds.
- Page 3, stream ID 436986: same as before, just with a shorter stalling period.
Other cases have different issues. For example, stream ID 34117 on page 3 is rather slow for most of the time and then suddenly gets faster at the end. However, it does not stall.
I do have tor logs and tor controller event logs for these cases. Here's a log containing many relevant STREAM and STREAM_BW events: https://people.torproject.org/~karsten/volatile/streams-2019-02-18.log.xz (61.1K)
These measurements have been made using tor versions 0.2.9.11-dev and 0.3.0.7-dev.
I can provide more data. But rather than uploading everything, please let me know what data would be most useful, and I'll provide just that.https://gitlab.torproject.org/tpo/core/tor/-/issues/29743Long-running tor instances fail to keep up-to-date directory information2022-06-17T18:54:59ZKarsten LoesingLong-running tor instances fail to keep up-to-date directory informationWe have a small number of long-running tor instances as part of our OnionPerf setups that are running 24/7. In the past, some of these tor instances got into a state where their directory information was no longer up-to-date enough to bu...We have a small number of long-running tor instances as part of our OnionPerf setups that are running 24/7. In the past, some of these tor instances got into a state where their directory information was no longer up-to-date enough to build circuits. In some cases they recovered after hours, days, or even weeks, but in some cases we had to restart the tor processes.
I'm attaching a graph that shows the number of open circuits as reported in heartbeat log messages. That number is relatively stable most of the time, depending on whether we're using the tor instance for making requests or for providing an onion service. But in some cases the number drops to zero, which coincides with the log message:
```
[notice] Our directory information is no longer up-to-date enough to build circuits: [...]
```
The graph also shows that sometimes the number magically goes up again. Those times coincide with the following log message:
```
[notice] We now have enough directory information to build circuits.
```
The purple dashed lines show when we restarted tor processes manually. Some of these restarts are unrelated to the number of open circuits. But some restarts happened explicitly because the tor instance was not working anymore for our measurements.
By the way, the op-nl instance shown in the middle was running 0.2.9.11-dev, whereas the op-us and op-hk instances were running 0.3.0.7-dev. It may be coincidence, but the older op-nl did not run out of up-to-date directory information, whereas the newer op-us and op-hk did. Was this issue maybe introduced in 0.3.0.x?
I have tor logs available for all these tor instances. I can easily provide them, either as a big tarball or for specific days and instances as a smaller tarball. Just let me know.