Trac issueshttps://gitlab.torproject.org/legacy/trac/-/issues2020-06-21T18:06:03Zhttps://gitlab.torproject.org/legacy/trac/-/issues/33034Gitlab soft deletes projects for 7 days2020-06-21T18:06:03ZCecylia BocovichGitlab soft deletes projects for 7 daysIt turns out, Gitlab doesn't allow you to delete public repositories immediately. They are soft-deleted (i.e., can be restored) for up to 7 days. This means that when we delete and try to recreate the same repository, our script fails be...It turns out, Gitlab doesn't allow you to delete public repositories immediately. They are soft-deleted (i.e., can be restored) for up to 7 days. This means that when we delete and try to recreate the same repository, our script fails because the name and path are already taken.
We should update the script to create new repositories for each upload. This can be done by modifying the repository name to be based on the tor browser release.Cecylia BocovichCecylia Bocovichhttps://gitlab.torproject.org/legacy/trac/-/issues/32498Consider updating MAR_CHANNEL_ID for nightly build (and maybe alpha too)2020-06-16T01:09:44ZboklmConsider updating MAR_CHANNEL_ID for nightly build (and maybe alpha too)In `browser/confvars.sh` (from `tor-browser.git`) we currently set `MAR_CHANNEL_ID` to `torbrowser-torproject-release` in all cases.
I see that Mozilla is using a different `MAR_CHANNEL_ID` for each of their channel (previously by updat...In `browser/confvars.sh` (from `tor-browser.git`) we currently set `MAR_CHANNEL_ID` to `torbrowser-torproject-release` in all cases.
I see that Mozilla is using a different `MAR_CHANNEL_ID` for each of their channel (previously by updating `browser/confvars.sh`, and now by setting it from taskcluster: https://hg.mozilla.org/releases/mozilla-release/rev/66f52bda7e14e26235bd0a43bb68ad11775046e4).
So I am wondering if we should do the same, and use a different `MAR_CHANNEL_ID` for nightly, and maybe for the alpha too.https://gitlab.torproject.org/legacy/trac/-/issues/30830Clean up snowflake broker logs2020-06-13T18:20:18ZCecylia BocovichClean up snowflake broker logsWe recently produces graphs from the unsanitized broker logs [here](https://trac.torproject.org/projects/tor/ticket/30693#comment:3). However, the script to produce these graphs was complicated due to the structure of the log messages. W...We recently produces graphs from the unsanitized broker logs [here](https://trac.torproject.org/projects/tor/ticket/30693#comment:3). However, the script to produce these graphs was complicated due to the structure of the log messages. We should evaluate log output from the broker and decide what to keep, what to change, and what to discard.Cecylia BocovichCecylia Bocovichhttps://gitlab.torproject.org/legacy/trac/-/issues/30579Add more STUN servers to the default snowflake configuration in Tor Browser2020-06-13T18:20:08ZCecylia BocovichAdd more STUN servers to the default snowflake configuration in Tor BrowserRight now snowflake blocking in China is happening in the client's connection to the default STUN server (which is set to Google's STUN servers). We should add more STUN servers, including ones that are popular in regions that are trying...Right now snowflake blocking in China is happening in the client's connection to the default STUN server (which is set to Google's STUN servers). We should add more STUN servers, including ones that are popular in regions that are trying to block snowflake so that blocking this stage causes more collateral damage.Cecylia BocovichCecylia Bocovichhttps://gitlab.torproject.org/legacy/trac/-/issues/34204Downgrade Travis stem version to a commit where tests pass.2020-06-13T15:53:32ZNick MathewsonDowngrade Travis stem version to a commit where tests pass.Due to https://github.com/torproject/stem/issues/63 our CI is failing. Let's downgrade to a working version of Stem unless it gets fixed right away.
I have a test PR at https://github.com/torproject/tor/pull/1889 ; if CI passes, I'll m...Due to https://github.com/torproject/stem/issues/63 our CI is failing. Let's downgrade to a working version of Stem unless it gets fixed right away.
I have a test PR at https://github.com/torproject/tor/pull/1889 ; if CI passes, I'll make PRs for the other branches and put this in needs_review.Tor: 0.4.4.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/33316Re-order early subsystem init order to match dependency order2020-06-13T15:51:25ZNick MathewsonRe-order early subsystem init order to match dependency orderIn #31634 (parent), we found :
>Our subsystem order does not currently match the topology very well at all: there are four discontinuities on the ordering. I think they are caused by: btrack, compress, winprocess, threads. We should re-...In #31634 (parent), we found :
>Our subsystem order does not currently match the topology very well at all: there are four discontinuities on the ordering. I think they are caused by: btrack, compress, winprocess, threads. We should re-order these, but doing so will require us to re-organize some of our code. That seems like a stability risk.
Let's try this in 0.4.4.Tor: 0.4.4.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/33014Refactor hs configuration parsing2020-06-13T15:50:05ZNick MathewsonRefactor hs configuration parsingWe accidentally duplicate a lot of our configuration parsing logic for our hs_config.c stuff. We should use our confmgt code instead, to detect duplicate options, malformed options, and so on.We accidentally duplicate a lot of our configuration parsing logic for our hs_config.c stuff. We should use our confmgt code instead, to detect duplicate options, malformed options, and so on.Tor: 0.4.4.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/32783Investigate clusterfuzz build failures2020-06-13T15:49:16ZNick MathewsonInvestigate clusterfuzz build failuresOur clusterfuzz setup has run into some build issues; I need to figure it out.Our clusterfuzz setup has run into some build issues; I need to figure it out.Tor: 0.4.3.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/31820Drop support for OpenSSL < 1.1.12020-06-13T15:49:13ZNick MathewsonDrop support for OpenSSL < 1.1.1As of 1 January 2020, there will be no supported OpenSSL version earlier than version 1.1.1. (See https://www.openssl.org/policies/releasestrat.html)
We can clean up our code by dropping backward-compatibility support for earlier versi...As of 1 January 2020, there will be no supported OpenSSL version earlier than version 1.1.1. (See https://www.openssl.org/policies/releasestrat.html)
We can clean up our code by dropping backward-compatibility support for earlier versions.
As we do this, we should test with supported versions of LibreSSL as well, since we try to support that too.Tor: unspecifiedNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/31898Occasional crash during shutdown when using Tor API2020-06-13T15:46:10ZMatthew FinkelOccasional crash during shutdown when using Tor APIThis is using a JNI wrapper.
I can probably create a test case for easier debugging. It seems like this is a race condition within the new pubsub system. I can reproduce this with an example program that configures Tor via the API and ...This is using a JNI wrapper.
I can probably create a test case for easier debugging. It seems like this is a race condition within the new pubsub system. I can reproduce this with an example program that configures Tor via the API and sends a `SIGNAL TERM` shortly after tor begins running and the control socket becomes available.
```
(gdb) bt
#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1 0x00007f8bcc822535 in __GI_abort () at abort.c:79
#2 0x00007f8b94b01dbe in tor_raw_abort_ () at src/lib/err/torerr.c:227
#3 0x00007f8b94b016d0 in crash_handler (sig=<optimized out>, si=<optimized out>, ctx_=<optimized out>) at src/lib/err/backtrace.c:175
#4 <signal handler called>
#5 0x00007f8b94ae0ae4 in max_in_sl (sl=<optimized out>, sl=<optimized out>, dflt=0) at src/lib/dispatch/dispatch_new.c:36
#6 dispatch_new (cfg=0x7f8b64033520) at src/lib/dispatch/dispatch_new.c:121
#7 0x00007f8b94ade9ac in pubsub_builder_finalize (builder=0x7f8b64029eb0, items_out=0x7f8b94c193b8 <the_pubsub_items>) at src/lib/pubsub/pubsub_build.c:293
#8 0x00007f8b9496666c in tor_mainloop_connect_pubsub (builder=0x7f8b64029eb0) at src/core/mainloop/mainloop_pubsub.c:56
#9 0x00007f8b94952741 in pubsub_install () at src/app/main/main.c:1246
#10 tor_run_main (tor_cfg=<optimized out>) at src/app/main/main.c:1297
#11 0x00007f8b9495034c in RunMain (env=0x7f8bc4250b18, thisObj=0x7f8b9431d9b8) at src/feature/api/TorApi.cc:179
#12 0x00007f8b949502dd in Java_api_TorApi_runMain (env=0x7f8bc4250b18, thisObj=0x7f8b9431d9b8) at src/feature/api/TorApi.cc:279
#13 0x00007f8bac8d4990 in ?? ()
#14 0x00007f8bac8ce450 in ?? ()
#15 0x0000000718c7c3a8 in ?? ()
#16 0x00007f8b9431d950 in ?? ()
#17 0x0000000000000000 in ?? ()
```
Bisecting found `a8c0f4ddfe3f0a63bd499959c8d921346aa9766e is the first bad commit`.
The fault is at the first dereference (i don't know if it's `*u` or `*maxptr`). I'm guessing the memory was already freed for exit by the time this procedure is called.
```
29 static int
30 max_in_sl(const smartlist_t *sl, int dflt)
31 {
32 uint16_t *maxptr = NULL;
33 SMARTLIST_FOREACH_BEGIN(sl, uint16_t *, u) {
34 if (!maxptr)
35 maxptr = u;
36 else if (*u > *maxptr)
37 maxptr = u;
38 } SMARTLIST_FOREACH_END(u);
39
40 return maxptr ? *maxptr : dflt;
41 }
```
Tor was configured with `--Log "notice stdout" --AvoidDiskWrites 1 --SocksPort unix:${HOME}/private_tor/test_socks --DisableNetwork 0`.
This shouldn't be a common edge case, but I stumbled upon it while testing something else.Tor: 0.4.1.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/31675Split microdescs_parse_from_string() into smaller functions2020-06-13T15:45:24ZNick MathewsonSplit microdescs_parse_from_string() into smaller functionsInstead of making an extended practracker exception here, we should make this function conform to our best practices.
I'm taking this on because I need a quick finger exercise between larger things.Instead of making an extended practracker exception here, we should make this function conform to our best practices.
I'm taking this on because I need a quick finger exercise between larger things.Tor: 0.4.2.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/31202Refactor practracker's issue-listing code to return a generator2020-06-13T15:43:43ZNick MathewsonRefactor practracker's issue-listing code to return a generatorRight now, this issue-listing code exposes the problems with "print()". Instead, it should yield them as a generator, so that the invoking code can decide what to do with them.Right now, this issue-listing code exposes the problems with "print()". Instead, it should yield them as a generator, so that the invoking code can decide what to do with them.Tor: 0.4.2.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/30307Make router_choose_random_node() linear instead of quadratic2020-06-13T15:41:03ZNick MathewsonMake router_choose_random_node() linear instead of quadraticSee parent for motivation.
The smartlist_subtract() function is O(m*n), so we should try not to use it here if we can.See parent for motivation.
The smartlist_subtract() function is O(m*n), so we should try not to use it here if we can.Tor: 0.4.1.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/28973Disable TLS1.3 when openssl bug 7712 is present2020-06-13T15:37:38ZNick MathewsonDisable TLS1.3 when openssl bug 7712 is presentSee #28616 for the impact of the bug in tor; see https://github.com/openssl/openssl/issues/7712 for the openssl issue.See #28616 for the impact of the bug in tor; see https://github.com/openssl/openssl/issues/7712 for the openssl issue.Tor: 0.3.3.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/28856Discard strcmp_len()2020-06-13T15:35:43ZNick MathewsonDiscard strcmp_len()We have a strcmp_len() function that we use extensively in parsecommon. But it's just a messed-up version of strncmp. We can get better performance here if we drop it.We have a strcmp_len() function that we use extensively in parsecommon. But it's just a messed-up version of strncmp. We can get better performance here if we drop it.Tor: 0.4.0.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/28853parse_short_policy() could be faster2020-06-13T15:35:42ZNick Mathewsonparse_short_policy() could be fasterThe main loop of parse_short_policy() is needlessly convoluted, and parses the same things more than once. It could be simplified greatly.The main loop of parse_short_policy() is needlessly convoluted, and parses the same things more than once. It could be simplified greatly.Tor: 0.4.0.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/28839Speed up parsing RSA onion keys from microdescriptors2020-06-13T15:35:35ZNick MathewsonSpeed up parsing RSA onion keys from microdescriptorsThe two functions `crypto_pk_read_public_key_from_string()` is about 9% of our startup time when we have directory information, and `router_set_rsa_onion_pkey()` is about 2.4%. There is a probably a good chance to save time here, in a f...The two functions `crypto_pk_read_public_key_from_string()` is about 9% of our startup time when we have directory information, and `router_set_rsa_onion_pkey()` is about 2.4%. There is a probably a good chance to save time here, in a few ways:
* Maybe pem_decode() could be faster. Right now it seems to be eating 3.53% of the startup time, and most of its energy is going into tor_asprintf().
* We possibly save a decode/encode cycle, since we are parsing asn1 into an RSA key, and then re-encoding that key into an asn1 string in router_set_rsa_onion_pkey().Tor: 0.4.0.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/28755Implement a K/V parser library2020-06-13T15:35:18ZDavid Gouletdgoulet@torproject.orgImplement a K/V parser libraryThe key/value format K=V is used extensively in the Tor code and seems many places uses their own way to parse that.
This ticket is for implementing a library to parse such K/V construction. There is a world where we might require V to ...The key/value format K=V is used extensively in the Tor code and seems many places uses their own way to parse that.
This ticket is for implementing a library to parse such K/V construction. There is a world where we might require V to be quoted or not so I'll let this challenge to the implementer(s).
This ticket comes from the work done in #28179 that implements messages from the PT to the Tor process.Tor: 0.4.0.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/28266Implement proposal 298-canonical-families.txt2020-06-13T15:33:35ZNick MathewsonImplement proposal 298-canonical-families.txtThis will improve the performance of #27359 by making more family lines match.This will improve the performance of #27359 by making more family lines match.Tor: 0.4.0.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/27892Split the non-stats part of the stats module into different modules2020-06-13T15:32:14ZNick MathewsonSplit the non-stats part of the stats module into different modulesPart of rephist.c is about predicted ports, which aren't really stats.
Part of geoip.c is about stats, but part is about mapping IPs to countries, which isn't stats.
Let's fix this while we can.Part of rephist.c is about predicted ports, which aren't really stats.
Part of geoip.c is about stats, but part is about mapping IPs to countries, which isn't stats.
Let's fix this while we can.Tor: 0.3.5.x-finalNick MathewsonNick Mathewson