The Tor Project issueshttps://gitlab.torproject.org/groups/tpo/-/issues2024-02-27T11:28:05Zhttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40327snowflake-01: Rotate snowflake-server.log2024-02-27T11:28:05ZLinus Nordberglinus@torproject.orgsnowflake-01: Rotate snowflake-server.log`/var/log/snowflake-server/snowflake-server.log` size is at 1.3G and should be rotated and compressed.
Should the process(es) writing to the file be informed somehow? Like what is done by logrotate(8) prerotate and postrotate.`/var/log/snowflake-server/snowflake-server.log` size is at 1.3G and should be rotated and compressed.
Should the process(es) writing to the file be informed somehow? Like what is done by logrotate(8) prerotate and postrotate.Linus Nordberglinus@torproject.orgLinus Nordberglinus@torproject.orghttps://gitlab.torproject.org/tpo/tpa/team/-/issues/41514metricsdb-01 is out of disk space on /2024-02-14T15:38:44ZKezmetricsdb-01 is out of disk space on /Roger reported metrics.tpo as being down (website returning 503). I checked nagios, and it looks like metricsdb-01 is out of disk space on the root partition. No other metrics-related issues are being reported in nagios, so I assume this...Roger reported metrics.tpo as being down (website returning 503). I checked nagios, and it looks like metricsdb-01 is out of disk space on the root partition. No other metrics-related issues are being reported in nagios, so I assume this is what's causing the metrics.tpo outage.HiroHirohttps://gitlab.torproject.org/tpo/core/arti/-/issues/1262Rethink descriptor publisher rate-limiting2024-02-01T18:10:55Zgabi-250Rethink descriptor publisher rate-limitingThe following discussion from !1951 should be addressed:
- [ ] @Diziet started a [discussion](https://gitlab.torproject.org/tpo/core/arti/-/merge_requests/1951#note_2991914): (+1 comment)
> So, suppose it's 50s since we last uploa...The following discussion from !1951 should be addressed:
- [ ] @Diziet started a [discussion](https://gitlab.torproject.org/tpo/core/arti/-/merge_requests/1951#note_2991914): (+1 comment)
> So, suppose it's 50s since we last uploaded. We reach this point and see that `duration_since_upload` is 50s, which is less than `UPLOAD_RATE_LIM_THRESHOLD` (60s).
>
> Then we call `start_rate_limit(60s)`. `start_rate_limit` calls `runtime.now()` and adds its argument, so scheduling a wakeup 60s from now.
>
> We will upload again 110s after the last upload. I think though, that we should do it 60s after.
>
> I think the root cause of this bug is the *storage* of a separate "we are rate limited" state in the reactor state, and using it to control the upload logic. Whether "we are rate limited" is really just "is the last upload more than `UPLOAD_RATE_LIM_THRESHOLD` ago" - ie, we could recalculate that on each loop iteration.
>
> In terms of `PublishStatus` (the status reporting output) I'm not sure "we are rate limited" is a particularly useful status to advertise. I think it's an entirely normal condition.
>
> Also we should perhaps randomise this?
>
> OTOH I don't think either of these questions are a blockers for this MR. The code here is a lot nicer, so thanks :-).Arti: Onion service supportgabi-250gabi-250https://gitlab.torproject.org/tpo/tpa/team/-/issues/41511upgrade crm-ext-01 to php 8 or retire2024-02-20T20:04:45Zanarcatupgrade crm-ext-01 to php 8 or retirein https://gitlab.torproject.org/tpo/tpa/team/-/issues/41252#note_2990644, @kez tested crm-ext-01 after I upgraded it and found the donate site completely broken by the PHP 8.2 upgrade. apparently, `implode` completely changed signature ...in https://gitlab.torproject.org/tpo/tpa/team/-/issues/41252#note_2990644, @kez tested crm-ext-01 after I upgraded it and found the donate site completely broken by the PHP 8.2 upgrade. apparently, `implode` completely changed signature in PHP and the old signature was dropped in PHP 8, which breaks a *lot* of things.
exactly how much is unclear, @kez estimated just the work to estimate that work to be a few hours of work.
for now i rolled back to the php 7.4 package from bullseye, and added it to the sources.list file (although puppet might have killed the .list file already). we need to figure out a plan to go forward, either port the code, or retire the box, which is the ultimate goal once donate-neo goes to production.Redesign donate.torproject.orghttps://gitlab.torproject.org/tpo/tpa/team/-/issues/41506Make it harder to forget to deploy tpo/web changes2024-01-30T15:03:48ZKezMake it harder to forget to deploy tpo/web changesIn tpo/web/tpo#403, I made a change, checked the review app, merged the change, checked the staging site, and *forgot* to deploy it to production. I only noticed because I was checking the pipelines for an unrelated issue and noticed the...In tpo/web/tpo#403, I made a change, checked the review app, merged the change, checked the staging site, and *forgot* to deploy it to production. I only noticed because I was checking the pipelines for an unrelated issue and noticed the deploy job didn't run. Bekeela also pointed out to me that the change wasn't deployed, but without the two of us checking, I never would've noticed and the change wouldn't have been deployed until the next time a change was needed on tpo.
I think it would really benefit us and our stakeholders to make it harder to forget to deploy to production. Maybe we could make a simple bot that checks if a change has been deployed to staging, but not production.Jérôme Charaouilavamind@torproject.orgJérôme Charaouilavamind@torproject.orghttps://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/42385Design dialog to share Lox invites2024-02-27T19:07:34ZJag TalonDesign dialog to share Lox invitescc @donuts @henry
**Design estimate:**
* Complexity: small (1 day)
* Copy existing modals from Firefox's design system.
* Uncertainty level: low (1.1)
* I believe there's no uncertainty here. All we need is something that's good ...cc @donuts @henry
**Design estimate:**
* Complexity: small (1 day)
* Copy existing modals from Firefox's design system.
* Uncertainty level: low (1.1)
* I believe there's no uncertainty here. All we need is something that's good enough for now.
* Total: 1-1.1 dayshttps://gitlab.torproject.org/tpo/network-health/metrics/descriptorParser/-/issues/80Makre sure we re-parse documents in case there are parsing errors2024-01-25T15:38:06ZGeorg KoppenMakre sure we re-parse documents in case there are parsing errorsOur download script is smart enough to re-download documents in case there were errors when fetching the latest ones, so we don't "lose" data in our DB:
```
download_url=https://collector.torproject.org/recent/$p/$u
log_file=$PAR...Our download script is smart enough to re-download documents in case there were errors when fetching the latest ones, so we don't "lose" data in our DB:
```
download_url=https://collector.torproject.org/recent/$p/$u
log_file=$PARSER_HOME/logs/downloads.log
if ! grep -q "$download_url" "$log_file"; then
status=$(wget --server-response ${download_url} 2>&1 | awk '/^ HTTP/{print $2}')
if [ "$status" = "200" ]; then
echo "$download_url" >> $log_file
fi
fi
```
However, we don't have a good solution for the missing data issue caused by a successful download yet parser errors. In that case the next download won't include the older documents for re-parsing (yet).https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/41069Unify the `start-$browser-browser` and the `$browser` scripts2024-01-25T14:34:45ZPier Angelo VendrameUnify the `start-$browser-browser` and the `$browser` scriptsCurrently, we ship two scripts: one is `start-tor-browser`/`start-mullvad-browser`, the other one is `firefox`/`mullvadbrowser`.
The reason seems to be related to the updater (passing the directory with the `libstdc++6` we ship to `LD_LI...Currently, we ship two scripts: one is `start-tor-browser`/`start-mullvad-browser`, the other one is `firefox`/`mullvadbrowser`.
The reason seems to be related to the updater (passing the directory with the `libstdc++6` we ship to `LD_LIBRARY_PATH` when needed).
However, some users might be launching `firefox` instead of `start-tor-browser` (or even worse, the actual binary - `firefox.real`!).
This is a risk, because they're missing home isolation and especially the fontconfig settings.
Could we do something to unify these scripts instead?
The first course of action would be to test Tor/Mullvad Browser (and the updater) in an old system, to trigger the need to use our libstdc++.https://gitlab.torproject.org/tpo/tpa/team/-/issues/41501retire individual grants in RT2024-01-25T03:44:00Zanarcatretire individual grants in RTin https://gitlab.torproject.org/tpo/tpa/team/-/issues/41496#note_2988546, @lavamind suggested we have a policy to only grant "groups" access and grant users access to those groups, to facilitate permissions management and auditing.
let...in https://gitlab.torproject.org/tpo/tpa/team/-/issues/41496#note_2988546, @lavamind suggested we have a policy to only grant "groups" access and grant users access to those groups, to facilitate permissions management and auditing.
let's do that.
@lavamind do you want this? or maybe @kez?https://gitlab.torproject.org/tpo/network-health/metrics/descriptorParser/-/issues/79Write contact information to `contact` (and not `contacts` field) when creati...2024-01-24T10:19:02ZGeorg KoppenWrite contact information to `contact` (and not `contacts` field) when creating a `server_status`Requires https://gitlab.torproject.org/tpo/network-health/metrics/metrics-sql-tables/-/issues/10.Requires https://gitlab.torproject.org/tpo/network-health/metrics/metrics-sql-tables/-/issues/10.https://gitlab.torproject.org/tpo/network-health/metrics/descriptorParser/-/issues/78Assemble `exit_policy_summary` and `exit_policy_v6_summary` in router status ...2024-01-24T10:17:09ZGeorg KoppenAssemble `exit_policy_summary` and `exit_policy_v6_summary` in router status if availableRequires https://gitlab.torproject.org/tpo/network-health/metrics/metrics-sql-tables/-/issues/9.Requires https://gitlab.torproject.org/tpo/network-health/metrics/metrics-sql-tables/-/issues/9.https://gitlab.torproject.org/tpo/network-health/metrics/metrics-sql-tables/-/issues/10Rename `contacts` to `contact` in our `server_status` table2024-01-24T10:19:03ZGeorg KoppenRename `contacts` to `contact` in our `server_status` tableThere are not several contacts to a router but just a single one. Let's use `contact` instead in our table.There are not several contacts to a router but just a single one. Let's use `contact` instead in our table.https://gitlab.torproject.org/tpo/network-health/metrics/metrics-sql-tables/-/issues/9Add `exit_policy_summary` and `exit_policy_v6_summary` entries to `server_sta...2024-01-24T10:17:10ZGeorg KoppenAdd `exit_policy_summary` and `exit_policy_v6_summary` entries to `server_status` tableFor our server status and our NetworkStatus API we miss `exit_policy_summary` and `exit_policy_v6_summary`.For our server status and our NetworkStatus API we miss `exit_policy_summary` and `exit_policy_v6_summary`.https://gitlab.torproject.org/tpo/network-health/metrics/descriptorParser/-/issues/77Assemble `dir_address` in a router status if available2024-01-24T09:53:03ZGeorg KoppenAssemble `dir_address` in a router status if availableRequires https://gitlab.torproject.org/tpo/network-health/metrics/metrics-sql-tables/-/issues/8.Requires https://gitlab.torproject.org/tpo/network-health/metrics/metrics-sql-tables/-/issues/8.https://gitlab.torproject.org/tpo/network-health/metrics/metrics-sql-tables/-/issues/8Add a `dir_address` column for `server_status` table2024-01-23T12:30:30ZGeorg KoppenAdd a `dir_address` column for `server_status` table`dir_port`s are still a thing, e.g. for directory authorities: `"DirPort" is its current directory port, or "0" for "none".`. Thus, we should add a `dir_address` column to our `server_status` table, so that we can provide the same output...`dir_port`s are still a thing, e.g. for directory authorities: `"DirPort" is its current directory port, or "0" for "none".`. Thus, we should add a `dir_address` column to our `server_status` table, so that we can provide the same output as Onionoo currently does. Either an IP address + port or it's omitted.https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/lyrebird/-/issues/40012Domain fronting requests don't work on some older Android versions2024-03-12T00:09:26ZPier Angelo VendrameDomain fronting requests don't work on some older Android versionsTor Browser for Android supports old versions of Android (API21, i.e., Android Lollipop).
While 13.5a3 doesn't work there because I used some NIO API that requires API26+, I've opened a MR to fix this (tpo/applications/tor-browser!894)....Tor Browser for Android supports old versions of Android (API21, i.e., Android Lollipop).
While 13.5a3 doesn't work there because I used some NIO API that requires API26+, I've opened a MR to fix this (tpo/applications/tor-browser!894).
While checking if things worked, I noticed that domain fronting requests don't (I don't get the special countries list).
As written in that MR, I tried to enable logging (I added `"-enableLogging", "-logLevel", "DEBUG", "-unsafeLogging"` as arguments), but I could get only these messages:
```
2024/01/22 10:20:23 [NOTICE]: obfs4proxy-0.0.14 - launched
2024/01/22 10:20:23 [INFO]: libObfs4proxy.so - initializing client transport listeners
2024/01/22 10:20:23 [INFO]: meek_lite - registered listener: 127.0.0.1:55852
2024/01/22 10:20:23 [INFO]: libObfs4proxy.so - accepting connections
2024/01/22 10:20:23 [WARN]: meek_lite(bridges.torproject.org:443) - closed connection: readfrom tcp 127.0.0.1:55852->127.0.0.1:48836: io: read/write on closed pipe
```
I think there might be some problems with some HTTPS certificate (at least letsencrypt had this problem a few years ago, indeed cohosh mentioned snowflake#40087. Fastly isn't using letsencrypt, but maybe they have a similar problem).
I can open bridges.torproject.org both in TBA and in the system browser, but I can't open https://moat.torproject.org.global.prod.fastly.net/ because it has a wrong certificate.
I don't think I'm using the latest version of Lyrebird, because in the last one the log file should be called lyrebird.log (I submitted a patch for that, unless I missed the log filename), but I can try to build one from a nightly build.Cecylia BocovichCecylia Bocovichhttps://gitlab.torproject.org/tpo/community/support/-/issues/40136Post a warning on the forum about the upcoming Win & macOS EOLs2024-02-15T14:41:01ZGusPost a warning on the forum about the upcoming Win & macOS EOLsTor Browser 14.0 (2024-Q4), based on Firefox ESR 128, will drop support for Windows 7, 8, & 8.1, and macOS 10.12, 10.13 & 10.14.
We should announce it on the Tor forum and Tor social channels.
* [ ] Draft a text to the forum
* [ ] Inf...Tor Browser 14.0 (2024-Q4), based on Firefox ESR 128, will drop support for Windows 7, 8, & 8.1, and macOS 10.12, 10.13 & 10.14.
We should announce it on the Tor forum and Tor social channels.
* [ ] Draft a text to the forum
* [ ] Inform Comms team about this
(cc @ebanam @nina @richard)https://gitlab.torproject.org/tpo/tpa/gitlab/-/issues/148Wrong domain of GitLab's mail server certificate2024-02-08T16:05:24ZMynacolWrong domain of GitLab's mail server certificateI wanted to reply to a GitLab issue by mail, but my mail server refused to send it, as the TLS certificate could not be verified. My mail server is configured to strictly verify the respective certificates.
The mail was headed to `[...]...I wanted to reply to a GitLab issue by mail, but my mail server refused to send it, as the TLS certificate could not be verified. My mail server is configured to strictly verify the respective certificates.
The mail was headed to `[...]@gitlab.torproject.org`. My mail server queried the MX record of gitlab.torproject.org, but only got a CNAME response, which leads to gitlab-02.torproject.org that points to the right IP addresses. Now my mail server expected a TLS certificate for gitlab.torproject.org, but your postfix provided a certificate for gitlab-02.torproject.org, which my mail server regarded as invalid.
The easiest way to fix this is to add a MX record to gitlab.torproject.org pointing at gitlab-02.torproject.org. That could even help with mail deliverability.
Alternatively, you can provide a certificate for gitlab.torproject.org from your mail server just like on the website.
Maybe the test page on [internet.nl](https://internet.nl/mail/gitlab.torproject.org/1127446/) helps you too.improve mail servicesJérôme Charaouilavamind@torproject.orgJérôme Charaouilavamind@torproject.orghttps://gitlab.torproject.org/tpo/tpa/team/-/issues/41482Automate renewal of self-signed LDAP cert2024-01-19T17:07:36ZKezAutomate renewal of self-signed LDAP certIn #41479 I renewed the self-signed LDAP cert for two years (730 days). That means that next time we renew it will be right after the holidays in 2026. It's not too much of a pain since it's only every 2 years, but it would be nice to no...In #41479 I renewed the self-signed LDAP cert for two years (730 days). That means that next time we renew it will be right after the holidays in 2026. It's not too much of a pain since it's only every 2 years, but it would be nice to not have to renew it right after we come back from our holiday break.
We could either automate the procedure entirely, or I could renew it again in a month or so so that the current cert will expire in February 2026. @anarcat any preferences or suggestions?Jérôme Charaouilavamind@torproject.orgJérôme Charaouilavamind@torproject.orghttps://gitlab.torproject.org/tpo/network-health/metrics/descriptorParser/-/issues/76extra_info_descriptor table default or wrong values2024-01-24T10:15:16Zjugaextra_info_descriptor table default or wrong valuesWorking on tpo/network-health/team#313, i thought to include whether a relay is overloaded in the grafana panels, so i tried this query:
```
select overload_ratelimits_version, overload_ratelimits_timestamp, overload_ratelimits_ratelimit...Working on tpo/network-health/team#313, i thought to include whether a relay is overloaded in the grafana panels, so i tried this query:
```
select overload_ratelimits_version, overload_ratelimits_timestamp, overload_ratelimits_ratelimit, overload_ratelimits_burstlimit , overload_ratelimits_read_count, overload_ratelimits_write_count, overload_fd_exhausted_version, overload_fd_exhausted_timestamp
from extra_info_descriptor
where fingerprint='0E13738FADDE15FC896E7CDB998C694F89F4E4B2';
```
which returns many rows as:
```
overload_ratelimits_version | overload_ratelimits_timestamp | overload_ratelimits_ratelimit | overload_ratelimits_burstlimit | overload_ratelimits_read_count | overload_ratelimits_write_count | overload_fd_exhausted_version | overload_fd_exhausted_timestamp
-----------------------------+-------------------------------+-------------------------------+--------------------------------+--------------------------------+---------------------------------+-------------------------------+---------------------------------
0 | 1969-12-31 23:59:59.999 | -1 | -1 | -1 | -1 | 0 | 1969-12-31 23:59:59.999
```
Maybe we could allow all these values to be NULL if the relay isn't overloaded?
I then tried to filter overload_ratelimits_read_count and overload_ratelimits_write_count without `-1` values and had to treat them as strings for that:
```
select count (*)
from extra_info_descriptor
where overload_ratelimits_write_count!='-1' and overload_ratelimits_read_count!='-1';
count
--------
169536
```
So maybe there's a bug here?
Maybe this was introduced by #47 (which i reviewed, ahem ;))
On a different issue, i think it'd be very helpful to add a VictoriaMetric metric to know whether a relay is overloaded.