The Tor Project issueshttps://gitlab.torproject.org/groups/tpo/-/issues2023-12-13T15:44:23Zhttps://gitlab.torproject.org/tpo/network-health/tor-weather/-/issues/83CI fails on main2023-12-13T15:44:23ZGeorg KoppenCI fails on mainCurrently CI fails on `main` with:
```
$ python3 -m tor_weather.job
Traceback (most recent call last):
File "/usr/local/lib/python3.9/runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/u...Currently CI fails on `main` with:
```
$ python3 -m tor_weather.job
Traceback (most recent call last):
File "/usr/local/lib/python3.9/runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/local/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/usr/local/lib/python3.9/site-packages/tor_weather/job.py", line 4, in <module>
entry_onionoo_job()
File "/usr/local/lib/python3.9/site-packages/tor_weather/__init__.py", line 54, in entry_onionoo_job
configure_database(app)
File "/usr/local/lib/python3.9/site-packages/tor_weather/__init__.py", line 82, in configure_database
db.init_app(app)
File "/usr/local/lib/python3.9/site-packages/flask_sqlalchemy/extension.py", line 355, in init_app
raise RuntimeError(
RuntimeError: Either 'SQLALCHEMY_DATABASE_URI' or 'SQLALCHEMY_BINDS' must be set.
Cleaning up project directory and file based variables 00:00
ERROR: Job failed: exit code 1
```Sarthik Guptasarthikg@icloud.comSarthik Guptasarthikg@icloud.comhttps://gitlab.torproject.org/tpo/web/manual/-/issues/134ci fails to build: jinja2.exceptions.TemplateNotFound: banner.html2022-11-15T01:50:41ZKezci fails to build: jinja2.exceptions.TemplateNotFound: banner.htmlhttps://gitlab.torproject.org/tpo/web/manual/-/jobs/198395
usually when this happens it's because `templates/banner.html` is a dangling symlink, so I'll take a look and see what happened.https://gitlab.torproject.org/tpo/web/manual/-/jobs/198395
usually when this happens it's because `templates/banner.html` is a dangling symlink, so I'll take a look and see what happened.https://gitlab.torproject.org/tpo/tpa/team/-/issues/40937CI for wiki-replica is broken2022-10-24T20:11:50ZJérôme Charaouilavamind@torproject.orgCI for wiki-replica is brokenSince the markdownlint project on GitHub has [updated](https://github.com/markdownlint/markdownlint/commit/865ab4408132de980baddb9448047f411f4e3325) their docker image a week ago, the [wiki-replica CI](https://gitlab.torproject.org/tpo/t...Since the markdownlint project on GitHub has [updated](https://github.com/markdownlint/markdownlint/commit/865ab4408132de980baddb9448047f411f4e3325) their docker image a week ago, the [wiki-replica CI](https://gitlab.torproject.org/tpo/tpa/wiki-replica/-/jobs) is unable to run any tests because the container bootstrap is failing with:
> ERROR: Job failed (system failure): Error response from daemon: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "/bin/bash": stat /bin/bash: no such file or directory: unknown (exec.go:78:0s)anarcatanarcathttps://gitlab.torproject.org/tpo/tpa/team/-/issues/40335CI is hitting DockerHub rate limits2023-10-12T18:26:50ZJim NewsomeCI is hitting DockerHub rate limitsOn runner ci-runner-x86-03-shadow I've managed to have DockerHub rate-limit our image pulls for a while: https://gitlab.torproject.org/jnewsome/sponsor-61-sims/-/jobs/27802
This makes me wonder whether the runner is caching Docker image...On runner ci-runner-x86-03-shadow I've managed to have DockerHub rate-limit our image pulls for a while: https://gitlab.torproject.org/jnewsome/sponsor-61-sims/-/jobs/27802
This makes me wonder whether the runner is caching Docker image layers - maybe [cache-dir](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runnersdocker-section) needs to be set?
Otherwise maybe DockerHub is throttling # of pull requests even when the individual layers end up being cached. That seems harder to solve, if so. I suppose we could change `pull_policy` to `if-not-present`, but then we'll have to be a bit careful to avoid/override using stale images.
Not sure how often this will be an issue in the steady state; at the time I hit the limit I was iterating pretty quickly on the script, so restarting the job every few minutes for a while. Unfortunately it doesn't appear trivial to set up a way to locally test.Jérôme Charaouilavamind@torproject.orgJérôme Charaouilavamind@torproject.orghttps://gitlab.torproject.org/tpo/core/arti/-/issues/1061CI, review use of after_script2023-10-14T23:37:19ZIan Jacksoniwj@torproject.orgCI, review use of after_scriptApparently `after_script` is strange. It seems like it probably doesn't do what we want, or, at least, what our use of it would seem to imply. See https://gitlab.torproject.org/tpo/tpa/team/-/issues/41323.
We should review our `.gitla...Apparently `after_script` is strange. It seems like it probably doesn't do what we want, or, at least, what our use of it would seem to imply. See https://gitlab.torproject.org/tpo/tpa/team/-/issues/41323.
We should review our `.gitlab-ci.yml`'s uses of `after_script` in the light of this information, and the gitlab docs (also linked to in that other ticket).trinity-1686atrinity-1686ahttps://gitlab.torproject.org/tpo/core/arti/-/issues/782CI-rejected TODO check2023-03-21T14:34:53ZIan Jacksoniwj@torproject.orgCI-rejected TODO checkI would like to have a thing we can put in our source tree that blocks merges. This is a very useful defence against a wide variety of mistakes (eg, mistakes where a debugging thing is left in).
@nickm and I had an irc conversation (be...I would like to have a thing we can put in our source tree that blocks merges. This is a very useful defence against a wide variety of mistakes (eg, mistakes where a debugging thing is left in).
@nickm and I had an irc conversation (below).
I think my plan is:
* Write script to reject reject `\bXXX+\b` (pcre case-sensitive)
* Fix up the violations in-tree (currently, there are 9) to be some other kind of TODO, or bugfix, or whatever
* Wire it into CI to run *in the last stage as its own job* so that you can iterate through CI with an XXX present.
```
16:48 <+Diziet> nickm: I was wondering if I might persuade to you that
something like this would be nice to have in the arti CI:
https://gitlab.torproject.org/Diziet/rust-derive-adhoc/-/blob/main/maint/check-blocking-todos
16:49 <+Diziet> Then when I put in `DROP THIS COMMIT` I can put an `// XXX` on
it too, preventing us from merging it by mistake
16:49 <+Diziet> Err, apparently I chose FIXME in d-a.
16:49 <+nickm> WFM. But I would prefer /XXXX*/ as the final string.
16:50 <+Diziet> Our existing codebase is a bit full of a mixture of XXX and
FIXME and TODO so we might need some discipline or to invent a
new tag.
16:50 <+nickm> I tried to use a discipline where XXXX is "this must be fixed.
Not to fix this is a bug." and TODO means "maybe someday"
16:50 <+Diziet> nickm: *Four* X's.
16:50 <+Diziet> (or more)
16:51 <+nickm> XXXX* matches 3 or more
16:51 <+Diziet> discipline> Very happy to adopt your convention, I just want
there to be one.
16:51 <+nickm> (I use 4 due to a supersition I got in early teenager years
about having my code misidentified as smut)
16:51 <+Diziet> Haha
16:51 <+Diziet> That's why I used FIXME in the d-a script
16:51 <+nickm> (The habit stuck.)
16:52 <+Diziet> OK, I will make a ticket and assign myself.
16:52 <+Diziet> zealot:arti> git-grep XXX | wc -l
16:52 <+Diziet> 10
16:52 <+nickm> one wrinkle is mksteemp
16:52 <+Diziet> Which is doable as a cleanup
16:52 <+nickm> another wrinkle is base64
16:52 <+Diziet> nickm: Yeah, but there's workarounds for that. Spell it with a
circumlocution.
16:53 <+Diziet> base64 can be broken up with a spurious " "
16:53 <+Diziet> FIXME is 5 characters so ~2^10x less likely to come up there...
16:54 <+Diziet> But
16:54 <+Diziet> zealot:arti> git-grep -i '\bFIXME\b' | wc -l
16:54 <+Diziet> 35
```Ian Jacksoniwj@torproject.orgIan Jacksoniwj@torproject.orghttps://gitlab.torproject.org/tpo/tpa/team/-/issues/40431ci-runner-01 invalid ubuntu package signatures2022-06-02T18:29:12ZJim Newsomeci-runner-01 invalid ubuntu package signaturesNot entirely sure what's happening here, but many of my jobs are getting invalid package signature errors from apt-get.
It *seems* to be happening on my jobs assigned to ci-runner-01 (such as https://gitlab.torproject.org/jnewsome/spons...Not entirely sure what's happening here, but many of my jobs are getting invalid package signature errors from apt-get.
It *seems* to be happening on my jobs assigned to ci-runner-01 (such as https://gitlab.torproject.org/jnewsome/sponsor-61-sims/-/jobs/39402), but not to the same jobs when assigned to other runners (such as https://gitlab.torproject.org/jnewsome/sponsor-61-sims/-/jobs/39420)
Maybe it's a problem with one of Ubuntu's package mirrors, and ci-runner-01 consistently gets the bad one?anarcatanarcathttps://gitlab.torproject.org/tpo/tpa/team/-/issues/40493ci-runner-01 out of space2022-06-02T18:28:48ZJim Newsomeci-runner-01 out of spacee.g. from https://gitlab.torproject.org/jnewsome/sponsor-61-sims/-/jobs/47427 :
```
$ apt-get install -y ca-certificates git
...
After this operation, 101 MB of additional disk space will be used.
E: You don't have enough free space in ...e.g. from https://gitlab.torproject.org/jnewsome/sponsor-61-sims/-/jobs/47427 :
```
$ apt-get install -y ca-certificates git
...
After this operation, 101 MB of additional disk space will be used.
E: You don't have enough free space in /builds/jnewsome/sponsor-61-sims/job-cache/apt/.
Cleaning up project directory and file based variables 00:01
ERROR: Job failed: exit code 1
```anarcatanarcathttps://gitlab.torproject.org/tpo/tpa/team/-/issues/40634ci-runner-01 running out of disk space2022-06-08T16:18:55Zanarcatci-runner-01 running out of disk spacewe've had a few of those warnings recently:
```
14:44:56 <nsa> tor-nagios: [ci-runner-01] disk usage on /srv is WARNING: DISK WARNING - free space: /srv 15772 MB (8% inode=80%):
```we've had a few of those warnings recently:
```
14:44:56 <nsa> tor-nagios: [ci-runner-01] disk usage on /srv is WARNING: DISK WARNING - free space: /srv 15772 MB (8% inode=80%):
```anarcatanarcathttps://gitlab.torproject.org/tpo/tpa/team/-/issues/40368ci-runner-x86-03-shadow can't resolve gitlab.torproject.org2021-08-27T17:16:53ZJim Newsomeci-runner-x86-03-shadow can't resolve gitlab.torproject.orgSome kind of network config issue?
From: https://gitlab.torproject.org/jnewsome/sponsor-61-sims/-/jobs/34654
```
Running with gitlab-runner 14.2.0 (58ba2b95)
on ci-runner-x86-03-shadow _6xksJ45
Preparing the "docker" executor 00:02
U...Some kind of network config issue?
From: https://gitlab.torproject.org/jnewsome/sponsor-61-sims/-/jobs/34654
```
Running with gitlab-runner 14.2.0 (58ba2b95)
on ci-runner-x86-03-shadow _6xksJ45
Preparing the "docker" executor 00:02
Using Docker executor with image ubuntu:20.04 ...
Pulling docker image ubuntu:20.04 ...
Using docker image sha256:1318b700e415001198d1bf66d260b07f67ca8a552b61b0da02b3832c778f221b for ubuntu:20.04 with digest ubuntu@sha256:82becede498899ec668628e7cb0ad87b6e1c371cb8a1e597d83a47fac21d6af3 ...
Preparing environment 00:00
Running on runner-6xksj45-project-969-concurrent-0 via chi-node-14...
Getting source from Git repository
Fetching changes with git depth set to 50...
Reinitialized existing Git repository in /builds/jnewsome/sponsor-61-sims/.git/
fatal: unable to access 'https://gitlab.torproject.org/jnewsome/sponsor-61-sims.git/': Could not resolve host: gitlab.torproject.org
```anarcatanarcathttps://gitlab.torproject.org/tpo/tpa/team/-/issues/40529ci-runner-x86-05 ran out of disk space2022-10-24T22:08:53Zanarcatci-runner-x86-05 ran out of disk space50G wasn't enough. the spec in #40500 mentioned "50GB >" so maybe we need to match what's on chi-node-14, which is, i believe, 250GB.
@jnewsome, how big should that /srv/shadow be anyways?50G wasn't enough. the spec in #40500 mentioned "50GB >" so maybe we need to match what's on chi-node-14, which is, i believe, 250GB.
@jnewsome, how big should that /srv/shadow be anyways?anarcatanarcathttps://gitlab.torproject.org/tpo/core/tor/-/issues/40049CID 1465290: Passing null pointer "ipv4_addr" to "geoip_get_country_by_addr"2020-07-16T18:17:57ZDavid Gouletdgoulet@torproject.orgCID 1465290: Passing null pointer "ipv4_addr" to "geoip_get_country_by_addr"```
*** CID 1465290: Null pointer dereferences (FORWARD_NULL)
/src/feature/nodelist/nodelist.c: 1991 in node_set_country()
1985 /* XXXXipv6 */
1986 if (node->rs)
1987 ipv4_addr = &node->rs->ipv4_addr;
1988 els...```
*** CID 1465290: Null pointer dereferences (FORWARD_NULL)
/src/feature/nodelist/nodelist.c: 1991 in node_set_country()
1985 /* XXXXipv6 */
1986 if (node->rs)
1987 ipv4_addr = &node->rs->ipv4_addr;
1988 else if (node->ri)
1989 ipv4_addr = &node->ri->ipv4_addr;
1990
>>> CID 1465290: Null pointer dereferences (FORWARD_NULL)
>>> Passing null pointer "ipv4_addr" to "geoip_get_country_by_addr", which dereferences it.
1991 node->country = geoip_get_country_by_addr(ipv4_addr);
1992 }
1993
1994 /** Set the country code of all routers in the routerlist. */
1995 void
1996 nodelist_refresh_countries(void)
```Tor: 0.4.5.x-freezeDavid Gouletdgoulet@torproject.orgDavid Gouletdgoulet@torproject.orghttps://gitlab.torproject.org/tpo/core/tor/-/issues/40263CID 1472584: Error handling issues in check_descriptor_ipaddress_changed()2021-02-12T17:54:21ZNick MathewsonCID 1472584: Error handling issues in check_descriptor_ipaddress_changed()It looks like Coverity wants us to check the return value on `relay_find_addr_to_publish`.
```
*** CID 1472584: Error handling issues (CHECKED_RETURN)
/src/feature/relay/router.c: 2700 in check_descriptor_ipaddress_changed()
2694 ...It looks like Coverity wants us to check the return value on `relay_find_addr_to_publish`.
```
*** CID 1472584: Error handling issues (CHECKED_RETURN)
/src/feature/relay/router.c: 2700 in check_descriptor_ipaddress_changed()
2694 previous = &my_ri->ipv6_addr;
2695 }
2696
2697 /* Attempt to discovery the publishable address for the family which will
2698 * actively attempt to discover the address if we are configured with a
2699 * port for the family. */
>>> CID 1472584: Error handling issues (CHECKED_RETURN)
>>> Calling "relay_find_addr_to_publish" without checking return value (as is done elsewhere 4 out of 5 times).
2700 relay_find_addr_to_publish(get_options(), family, RELAY_FIND_ADDR_NO_FLAG,
2701 ¤t);
2702
```
Assigning to @dgoulet, but please let me know if you want me to do this.Tor: 0.4.5.x-stableDavid Gouletdgoulet@torproject.orgDavid Gouletdgoulet@torproject.orghttps://gitlab.torproject.org/tpo/core/tor/-/issues/40442CID 1489776: Questionable sign extension in congestion control logic2021-10-21T12:41:54ZNick MathewsonCID 1489776: Questionable sign extension in congestion control logicLooks like we're missing signed and unsigned here unintentionally. Let's resolve this by adding explicit casts as necessary so that we don't have to reason too hard about C's cursed integer promotion logic.
```
/src/core/or/congestion...Looks like we're missing signed and unsigned here unintentionally. Let's resolve this by adding explicit casts as necessary so that we don't have to reason too hard about C's cursed integer promotion logic.
```
/src/core/or/congestion_control_common.c: 789 in congestion_control_update_circuit_bdp()
783 uint64_t delta = now_usec - timestamp_usec;
784
785 /* The acked data is in sendme_cnt-1 chunks, because we are counting the
786 * data that is processed by the other endpoint *between* all of these
787 * sendmes. There's one less gap between the sendmes than the number
788 * of sendmes. */
>>> CID 1489776: Integer handling issues (SIGN_EXTENSION)
>>> Suspicious implicit sign extension: "cc->sendme_inc" with type "uint8_t" (8 bits, unsigned) is promoted in "(sendme_cnt - 1) * cc->sendme_inc" to type "int" (32 bits, signed), then sign-extended to type "unsigned long" (64 bits, unsigned). If "(sendme_cnt - 1) * cc->sendme_inc" is greater than 0x7FFFFFFF, the upper bits of the result will all be 1.
789 uint64_t cells = (sendme_cnt-1)*cc->sendme_inc;
790
791 /* The bandwidth estimate is cells/delta, which when multiplied
792 * by min RTT obtains the BDP. However, we multiply first to
793 * avoid precision issues with the RTT being close to delta in size. */
794 sendme_rate_bdp = cells*cc->min_rtt_usec/delta;
```
cc: @mikeperry @dgouletTor: 0.4.7.x-freezeMike PerryMike Perryhttps://gitlab.torproject.org/tpo/core/arti/-/issues/50Circmgr needs a significant backend revision2021-06-10T12:02:13ZNick MathewsonCircmgr needs a significant backend revisionThe existing circmgr implementation doesn't always do the right thing. It keeps circuits and pending circuits in the same data structure. When a pending circuit is done, everything that was waiting for it is notified... but other strea...The existing circmgr implementation doesn't always do the right thing. It keeps circuits and pending circuits in the same data structure. When a pending circuit is done, everything that was waiting for it is notified... but other streams that were waiting for a different port which the same circuit happens to handle are not notified.
We should have a new implementation that keeps a list of waiting requests, and launches circuits as appropriate to meet those streams' needs. When a circuit is done, it should notify every request that would be satisfied by it.
This doesn't have to be done for milestone A1, thankfully.Arti 0.0.1 release: basic anonymityNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/arti/-/issues/656CircMgr: retire_circuits() does not affect pending circuits.2022-11-22T19:00:21ZNick MathewsonCircMgr: retire_circuits() does not affect pending circuits.From #650.
I've found a bug in the existing "retire circuits" code: it does not affect in-progress circuits. Those circuits can still succeed, and if they do, they will announce themselves to any pending circuit requests.
It's a bit tr...From #650.
I've found a bug in the existing "retire circuits" code: it does not affect in-progress circuits. Those circuits can still succeed, and if they do, they will announce themselves to any pending circuit requests.
It's a bit tricky to avoid a race condition here.
We could either:
* Add a way to cancel in-progress pending circuits.
* Add a generation counter to the circuit manager, so that an earlier-generation circuit is never a good answer for the current generation.
I kind of lean towards the second as easier to reason about.Arti 1.1.0: Anticensorship readyNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/arti/-/issues/650CircMgr::reconfigure must look at bridge configuration2022-11-22T19:00:21ZNick MathewsonCircMgr::reconfigure must look at bridge configurationIn earlier design discussions, we decided that we didn't want to close all our existing channels and circuits when we changed our bridges, or we changed from using bridges to not using bridges. That's cool.
We should, however, tell the...In earlier design discussions, we decided that we didn't want to close all our existing channels and circuits when we changed our bridges, or we changed from using bridges to not using bridges. That's cool.
We should, however, tell the CircMgr that the circuits it was previously using might not be usable any more:
* a non-bridge circuit is non-usable if we start using bridges.
* a bridge circuit is non-usable if we stop using bridges, or if we remove that bridge from our configuration.
We have some logic to handle a similar case already: see `retire_all_circuits()` and its use in `CircMgr::reconfigure`. See also the `TODO(nickm)` comment in `retire_all_circuits()`.Arti 1.1.0: Anticensorship readyNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/applications/vpn/-/issues/37Circuit display2023-06-20T20:49:19Zmicahmicah@torproject.orgCircuit displaySimilar to tor browser, users would like to see the circuit each app is being routed through, and view each relay’s IP address and location.Similar to tor browser, users would like to see the circuit each app is being routed through, and view each relay’s IP address and location.VPN pre-alpha 02cybertacybertahttps://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/41980Circuit display headline is misaligned in 13.0 alpha2023-10-03T13:28:19ZdonutsCircuit display headline is misaligned in 13.0 alphaThe headline reading "Circuit display for <website>" is vertically misaligned in 13.0 alpha. See Firefox's "Site information" wingpanel for the correct alignment.The headline reading "Circuit display for <website>" is vertically misaligned in 13.0 alpha. See Firefox's "Site information" wingpanel for the correct alignment.https://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/42045Circuit panel overflows with long ipv6 addresses2023-10-03T13:27:48ZhenryCircuit panel overflows with long ipv6 addresseswith esr115, some ip addresses in the circuit panel now overflow the panel width, without growing the panel as expected.
This is a regression from moving to esr115 since the moz-box layout now uses the flex layout.with esr115, some ip addresses in the circuit panel now overflow the panel width, without growing the panel as expected.
This is a regression from moving to esr115 since the moz-box layout now uses the flex layout.henryhenry