The Tor Project issueshttps://gitlab.torproject.org/groups/tpo/-/issues2022-06-03T16:40:03Zhttps://gitlab.torproject.org/tpo/core/tor/-/issues/40620XOFF notice at edge connection2022-06-03T16:40:03ZDavid Gouletdgoulet@torproject.orgXOFF notice at edge connectionThis Exit relay operator just informed us (on #tor-relays) that their Exit has seen this at least 979 times (duplicates hidden):
> Request to start reading on an edgeconn blocked with XOFF
https://metrics.torproject.org/rs.html#details...This Exit relay operator just informed us (on #tor-relays) that their Exit has seen this at least 979 times (duplicates hidden):
> Request to start reading on an edgeconn blocked with XOFF
https://metrics.torproject.org/rs.html#details/B916FA873E00A1A1084DBA40DD767FB659BEA367
Also appears to be discussed in: http://eweiibe6tdjsdprb4px6rqrzzcsi22m4koia44kc5pcjr7nec2rlxyad.onion/tpo/core/tor/-/issues/40612#note_2806954Tor: 0.4.7.x-post-stableMike PerryMike Perryhttps://gitlab.torproject.org/tpo/network-health/metrics/website/-/issues/40047userstats-bridge-country graph is undercounting users2022-10-26T15:31:27ZHirouserstats-bridge-country graph is undercounting usersWe are undercounting bridge users by about 14,000 or 14%, since only 12.5% of Snowflake users are being counted. (https://gitlab.torproject.org/tpo/network-health/metrics/onionoo/-/issues/40022#note_2793712)We are undercounting bridge users by about 14,000 or 14%, since only 12.5% of Snowflake users are being counted. (https://gitlab.torproject.org/tpo/network-health/metrics/onionoo/-/issues/40022#note_2793712)Metrics OKRs Q3-Q4 2022HiroHirohttps://gitlab.torproject.org/tpo/network-health/onbasca/-/issues/113checkmanifest check is failing2023-10-05T19:22:44ZGeorg Koppencheckmanifest check is failing```
tox -e checkmanifest
checkmanifest installed: build==0.7.0,check-manifest==0.47,packaging==21.3,pep517==0.12.0,pyparsing==3.0.7,toml==0.10.2,tomli==2.0.1
checkmanifest run-test-pre: PYTHONHASHSEED='1414042973'
checkmanifest run-test:...```
tox -e checkmanifest
checkmanifest installed: build==0.7.0,check-manifest==0.47,packaging==21.3,pep517==0.12.0,pyparsing==3.0.7,toml==0.10.2,tomli==2.0.1
checkmanifest run-test-pre: PYTHONHASHSEED='1414042973'
checkmanifest run-test: commands[0] | check-manifest
lists of files in version control and sdist do not match!
missing from sdist:
LICENSES/BSD-3-Clause.txt
LICENSES/CC0-1.0.txt
suggested MANIFEST.in rules:
recursive-include LICENSES *.txt
ERROR: InvocationError for command /home/gk/nethealth/onbasca/.tox/checkmanifest/bin/check-manifest (exited with code 1)
______________________________________________________________________________ summary _______________________________________________________________________________
ERROR: checkmanifest: commands failed
```
That's on main + !26. Do we need to update our CI to catch those things?onbasca: 1.0Georg KoppenGeorg Koppenhttps://gitlab.torproject.org/tpo/network-health/onbasca/-/issues/108reuse check is failing2023-10-05T19:22:44ZGeorg Koppenreuse check is failingOur `python39tormaster` job is failing:
```
# MISSING COPYRIGHT AND LICENSING INFORMATION
The following files have no copyright and licensing information:
* MANIFEST.in
# SUMMARY
* Bad licenses:
* Deprecated licenses:
* Licenses with...Our `python39tormaster` job is failing:
```
# MISSING COPYRIGHT AND LICENSING INFORMATION
The following files have no copyright and licensing information:
* MANIFEST.in
# SUMMARY
* Bad licenses:
* Deprecated licenses:
* Licenses without file extension:
* Missing licenses:
* Unused licenses:
* Used licenses: BSD-3-Clause, CC0-1.0
* Read errors: 0
* Files with copyright information: 110 / 111
* Files with license information: 110 / 111
Unfortunately, your project is not compliant with version 3.0 of the REUSE Specification :-(
ERROR: InvocationError for command /builds/tpo/network-health/onbasca/.tox/reuse/bin/reuse lint (exited with code 1)
```onbasca: 1.0jugajugahttps://gitlab.torproject.org/tpo/network-health/sbws/-/issues/40128Bump debian package to v1.4.02022-03-04T08:15:18ZjugaBump debian package to v1.4.0sbws: 1.3.x-finaljugajugahttps://gitlab.torproject.org/tpo/tpa/team/-/issues/40633Migrate Puppet sudo configs to saz/sudo module2023-04-12T17:23:44ZJérôme Charaouilavamind@torproject.orgMigrate Puppet sudo configs to saz/sudo moduleCurrently in the `tor-puppet` repository, most `sudo` configurations are deployed via a single file resource that's identical across all our machines. This is less than ideal from a security standpoint since this one file reveals a great...Currently in the `tor-puppet` repository, most `sudo` configurations are deployed via a single file resource that's identical across all our machines. This is less than ideal from a security standpoint since this one file reveals a great deal about the permissions granted to users on our different machines. Furthermore, the file is deployed without any syntax checking: a single error in there is susceptible to break `sudo` for many users and systems.
I propose we deploy the 3rd party [saz/puppet-sudo](https://github.com/saz/puppet-sudo) module in our Puppet infrastructure and leverage this along with a new `profile::sudo` class to progressively migrate from the current `sudo` module.
The deployment would be staged in several steps:
* [x] audit [saz/puppet-sudo](https://github.com/saz/puppet-sudo)
* [ ] ~~rename the current `sudo` module to `sudo_legacy` and modify the base includes to use the new class name~~
* [X] pull in the new sudo module in `3rdparty/modules`
* [X] add a new `profile::sudo` class to create `sudo::conf` resources from Hiera data
* [x] move the sudoers-`file` resource in `role::network_health_relay` to Hiera
* [ ] ~~validate that `profile::sudo` and `sudo_legacy` can coexist~~
* [x] add `profile::sudo` to base classes and deploy progressively
* [x] move sudoers-`file` resource in other classes to Hiera
* [x] retire `sudo_legacy`
* [ ] ~~progressively convert monolithic `sudoers` to Hiera~~ move to a separate ticketcleanup and publish the sysadmin codebaseJérôme Charaouilavamind@torproject.orgJérôme Charaouilavamind@torproject.orghttps://gitlab.torproject.org/tpo/tpa/team/-/issues/40627consider disabling sender verification2022-04-04T19:03:06Zanarcatconsider disabling sender verificationUsers of the tails project are complaining that they cannot receive notifications from weblate server on their `@torproject.org` email address because of sender verification ([`reject_unverified_sender`](http://www.postfix.org/postconf.5...Users of the tails project are complaining that they cannot receive notifications from weblate server on their `@torproject.org` email address because of sender verification ([`reject_unverified_sender`](http://www.postfix.org/postconf.5.html#reject_unverified_sender) in Postfix). here's a part of a bounce they typically receive:
```
<XXXX@torproject.org>: host eugeni.torproject.org[49.12.57.136] said: 450
4.1.7 <weblate@translate.tails.boum.org>: Sender address rejected:
unverified address: host lizard.tails.boum.org[204.13.164.63] said: 554
5.7.1 <weblate@translate.tails.boum.org>: Relay access denied (in reply to
RCPT TO command) (in reply to RCPT TO command)
[ message/delivery-status ]
Reporting-MTA: dns; lizard.tails.boum.org
X-Postfix-Queue-ID: D121A4189A
X-Postfix-Sender: rfc822; weblate@translate.tails.boum.org
Arrival-Date: Thu, 17 Feb 2022 03:03:17 +0000 (UTC)
Final-Recipient: rfc822; XXXX@torproject.org
Original-Recipient: rfc822;XXXX@torproject.org
Action: delayed
Status: 4.1.7
Remote-MTA: dns; eugeni.torproject.org
Diagnostic-Code: smtp; 450 4.1.7 <weblate@translate.tails.boum.org>: Sender
address rejected: unverified address: host
lizard.tails.boum.org[204.13.164.63] said: 554 5.7.1
<weblate@translate.tails.boum.org>: Relay access denied (in reply to RCPT
TO command)
Will-Retry-Until: Tue, 22 Feb 2022 03:03:17 +0000 (UTC)
[ text/rfc822-headers ]
Return-Path: <weblate@translate.tails.boum.org>
Received: from 55e085888c8e (localhost [127.0.0.1])
by lizard.tails.boum.org (Postfix) with ESMTP id D121A4189A
for <XXXX@torproject.org>; Thu, 17 Feb 2022 03:03:17 +0000 (UTC)
Content-Type: multipart/related;
boundary="===============8284330724149343160=="
MIME-Version: 1.0
```
we should consider disabling this check or allowing tail's weblate to bypass it.
if we do disable it system-wide, we should consider the impact it will have on incoming spam, as that probably stops a lot of abuse already... this might something we have stats on in prometheus.improve mail servicesanarcatanarcathttps://gitlab.torproject.org/tpo/core/onionmasq/-/issues/1Build Android prototype app2023-09-05T15:28:28ZAlexander Færøyahf@torproject.orgBuild Android prototype appAn initial prototype should support the following operations:
1. Have a text view field, that is copyable, but not editable, where we can log text output to. This will allow us to copy log data from onionmasq to emails/pad/whatever for ...An initial prototype should support the following operations:
1. Have a text view field, that is copyable, but not editable, where we can log text output to. This will allow us to copy log data from onionmasq to emails/pad/whatever for sharing/debugging purposes.
2. A button to connect and disconnect the VPN.
3. It would be useful to have a list of applications that are present on the device where the user can select/unselect individual items. The application need to convert these application names to Android UID's (so a vector of integer?) that we can pass to onionmasq for filtering purposes.
4. @ahf will be working on figuring out the FFI layer between the app and the Rust code, but for now the important thing is being able to pass the VPN file descriptor from Android's VPN subsystem to Rust.Onionmasq 1.0.0: Initial releasecybertacybertahttps://gitlab.torproject.org/tpo/tpa/team/-/issues/40622deploy postfix-based incoming DKIM verification on polyanthum2022-02-28T16:15:23Zanarcatdeploy postfix-based incoming DKIM verification on polyanthumas part of #40581, we deployed a new transport to deliver mails directly to the bridges mail server from our postfix server. but in doing so, we dropped DKIM verification, which was piped in by procmail.
i don't want to bring procmail b...as part of #40581, we deployed a new transport to deliver mails directly to the bridges mail server from our postfix server. but in doing so, we dropped DKIM verification, which was piped in by procmail.
i don't want to bring procmail back, but we should restore DKIM functionality here. and this would serve as a good test bed for the broader "plan for email" (#40363, %"improve mail services")
@meskio, could you let me know when the test setup is expected to enter production, to get an idea of the timeline here? thanks!improve mail servicesanarcatanarcathttps://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/40773Update the about:torconnect frontend page to match additional UI flows2022-04-13T06:35:32ZrichardUpdate the about:torconnect frontend page to match additional UI flows- UI specs: https://www.figma.com/file/Vsh1aPOZGneDX4Zp27mjsK/torconnect?node-id=531%3A2047
- User flow: https://www.figma.com/file/Vsh1aPOZGneDX4Zp27mjsK/torconnect?node-id=531%3A2047- UI specs: https://www.figma.com/file/Vsh1aPOZGneDX4Zp27mjsK/torconnect?node-id=531%3A2047
- User flow: https://www.figma.com/file/Vsh1aPOZGneDX4Zp27mjsK/torconnect?node-id=531%3A2047Tor Browser 11.5Pier Angelo VendramePier Angelo Vendramehttps://gitlab.torproject.org/tpo/network-health/sbws/-/issues/40119gabelmoo's sbws weight sum is greater than the 50% of the consensus2023-03-17T11:43:50Zjugagabelmoo's sbws weight sum is greater than the 50% of the consensusAs explained at #40115 gabelmoo's sbws is reporting this warning.
```
It looks like gabelmoo's had always greatest consensus weight sum (see for instance the total consensus weight graph back in 2018 when the log was created: https://met...As explained at #40115 gabelmoo's sbws is reporting this warning.
```
It looks like gabelmoo's had always greatest consensus weight sum (see for instance the total consensus weight graph back in 2018 when the log was created: https://metrics.torproject.org/totalcw.png?start=2018-10-01&end=2018-10-31), what it probably due gabelmoo's location (Germany). What i'm not sure is whether it was as much as the 50% before because relays were getting stuck in bandwidth partitions with torflow or we just didn't notice cause torflow wasn't logging that.
```
This issue is to investigate why this is happening.sbws: 1.3.x-finaljugajugahttps://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/40742Remove workaround for fixing --disable-maintenance-service build bustage2021-12-15T18:53:57ZGeorg KoppenRemove workaround for fixing --disable-maintenance-service build bustageOur rambling cypherpunk pointed out that we actually can remove the cherry-picked WIP patch as the real patches for https://bugzilla.mozilla.org/show_bug.cgi?id=1728536 landed on 91.4.0esr.Our rambling cypherpunk pointed out that we actually can remove the cherry-picked WIP patch as the real patches for https://bugzilla.mozilla.org/show_bug.cgi?id=1728536 landed on 91.4.0esr.Tor Browser 11.5Georg KoppenGeorg Koppenhttps://gitlab.torproject.org/tpo/network-health/sbws/-/issues/40112Figure out why gabelmoo is failing to measure so many relays2022-01-21T16:06:22ZjugaFigure out why gabelmoo is failing to measure so many relaysFrom irc conversation:
```
[08:34] < juga> hmm, it has measured ~7000 relays, but for some reason it failed with ~3000
[...]
[09:11] < juga> Sebastian: GeKo: one possible reason i was thinking on, is that some exits don't allow to exit t...From irc conversation:
```
[08:34] < juga> hmm, it has measured ~7000 relays, but for some reason it failed with ~3000
[...]
[09:11] < juga> Sebastian: GeKo: one possible reason i was thinking on, is that some exits don't allow to exit to places in the same country and many exits
are in DE, but that doesn't seem to be the case. Sebastian: is it possible that the web server is not managing well many requests? (most of
them time out)
[09:13] < juga> though if it's the same torflow was using, it shouldn't be the case either...
```sbws: 1.3.x-finaljugajugahttps://gitlab.torproject.org/tpo/network-health/sbws/-/issues/40111Figure out why Faravahar seems to report more relays every day around 002022-03-28T13:53:03ZjugaFigure out why Faravahar seems to report more relays every day around 00This can be observed at https://consensus-health.torproject.org/graphs.html (BWAuth Measured Relays):
![Screenshot_from_2021-10-22_13-17-25](/uploads/97a63a518b334420ae10bda1d699a749/Screenshot_from_2021-10-22_13-17-25.png)This can be observed at https://consensus-health.torproject.org/graphs.html (BWAuth Measured Relays):
![Screenshot_from_2021-10-22_13-17-25](/uploads/97a63a518b334420ae10bda1d699a749/Screenshot_from_2021-10-22_13-17-25.png)sbws: 1.3.x-finalGeorg KoppenGeorg Koppenhttps://gitlab.torproject.org/tpo/tpa/team/-/issues/40425improve the spam filter on the RT frontdesk2023-07-04T16:02:16Zchampionquizzerchampionquizzer@torproject.orgimprove the spam filter on the RT frontdeskOn frontdesk, we are consistently receiving nearly 1200-1300 spam tickets per month. Recently (since 17th September, 2021), we have started blocking certain recurring spammers on a user level, yet the statistics don't seem so promising. ...On frontdesk, we are consistently receiving nearly 1200-1300 spam tickets per month. Recently (since 17th September, 2021), we have started blocking certain recurring spammers on a user level, yet the statistics don't seem so promising. Since 18th September 00:00 UTC we have received 270 tickets in spam (and there's much more in the frontdesk queue which we haven't yet manually queued as spam). Would it be possible to do something on your end? :)
Thanks!improve mail servicesJérôme Charaouilavamind@torproject.orgJérôme Charaouilavamind@torproject.orghttps://gitlab.torproject.org/tpo/network-health/sbws/-/issues/40102CI failing with some tor deb packages version2021-12-14T10:11:30ZjugaCI failing with some tor deb packages versionWe should check which are the correct names.
For instance: Job [#36265](https://gitlab.torproject.org/tpo/network-health/sbws/-/jobs/36265) failed for 39e14543614efbaf993583a63fdd967a7e83d1e1:
```
E: Release 'tor-nightly-master-bullseye...We should check which are the correct names.
For instance: Job [#36265](https://gitlab.torproject.org/tpo/network-health/sbws/-/jobs/36265) failed for 39e14543614efbaf993583a63fdd967a7e83d1e1:
```
E: Release 'tor-nightly-master-bullseye' for 'tor' was not found
```sbws: 1.3.x-finaljugajugahttps://gitlab.torproject.org/tpo/tpa/team/-/issues/40363TPA-RFC-15: plan regarding mail standards (DKIM,SPF, DMARC)2022-12-01T20:21:18ZanarcatTPA-RFC-15: plan regarding mail standards (DKIM,SPF, DMARC)i'm thinking of making a formal TPA-RFC to adopt SPF, DKIM and DMARC formally, and set a deployment plan on how to do this effectively. we can't, for example, deploy DKIM and SPF on @torproject.org without first providing a way for peopl...i'm thinking of making a formal TPA-RFC to adopt SPF, DKIM and DMARC formally, and set a deployment plan on how to do this effectively. we can't, for example, deploy DKIM and SPF on @torproject.org without first providing a way for people to submit email through there, nor can we deploy DKIM on lists.tpo without first fixing DMARC handling on outgoing email.
this needs more work than what we currently have laid out in the roadmap.
this will probably require a TPA-RFC process (because it's a fairly radical change), but so far I'm just brainstorming here.
known issues:
* [Yahoo](https://gitlab.torproject.org/tpo/tpa/team/-/issues/34134), [state.gov](https://gitlab.torproject.org/tpo/tpa/team/-/issues/40202), [Gmail](https://gitlab.torproject.org/tpo/tpa/team/-/issues/40170), [Gmail again](https://gitlab.torproject.org/tpo/tpa/team/-/issues/40149) (from the roadmap item)
* Civi lacking entries (tpo/web/donate-static#15)
* complaints about lists.tpo lacking SPF/DKIM (https://gitlab.torproject.org/tpo/tpa/team/-/issues/40347)
* DMARC and lists.tpo (https://gitlab.torproject.org/tpo/tpa/team/-/issues/19914)
interlocking issues:
* the submission mail server (https://gitlab.torproject.org/tpo/tpa/team/-/issues/30608) requires upgrades and changes to ud-ldap #40062 and #40182
* general SPF deployment requires the submission mail server
* DKIM on lists.tpo (https://gitlab.torproject.org/tpo/tpa/team/-/issues/40347) requires DMARC workarounds (https://gitlab.torproject.org/tpo/tpa/team/-/issues/40347)
* general DKIM deployment requires testing (e.g. on tpo/web/donate-static#15) and integration with DNS (how?)
* general DKIM deployment requires the submission mail server as well
* SPF and DKIM require DMARC to properly function
* DMARC requires a monitoring system to be effectively enabled (otherwise you might break legitimate emails going out and never know about it)
* in any case, we need end-to-end deliverability tests to see if measures we take have an impact, see tpo/tpa/team#40494
The current situation is:
* no DKIM deployment, except bridgedb which *verifies* incoming DKIM signatures (*but* does nothing with it, actually)
* no SPF deployment, except on lists.tpo and crm.tpo
* no DMARC deployment or reporting
* no end-to-end deliverability testing, other than CiviCRM and Mailman internal monitoring systems and word-of-mouth
* most servers relay their mails through eugeni except bridges, gettor, GitLab, db.tpo (alberti), RT, submit-01 and CiviCRM (according to the profile::postfix Puppet class setting, which is also enabled by the profile::dovecot::private)
* we run restricted or "private" Dovecot servers (restricted to a single user) on only two servers (GitLab, CiviCRM), and such a configuration could be used to receive DMARC reports, for example... we also run an authentication-only (no mailbox or IMAP) dovecot for SASL authentication in postfix (on `submit-01`)
the task list here is:
* [x] prepare a draft RFC to discuss inside TPA (done: TPA-RFC-15: https://gitlab.torproject.org/tpo/tpa/team/-/wikis/policy/tpa-rfc-15-email-services)
* [x] complete the RFC after review (done, all TODO checked)
* [x] review this tickets for comments
* [x] do a final edit
* [x] send to tor-internal for discussion
* [x] make slides for the all hands presentation
* [x] do a trial run of the presentation
* [x] present a summary at the all hands
* [x] another round of reviews after comments
* [ ] followup on the decision after this (probably adding and sorting issues in %"improve mail services" )improve mail servicesanarcatanarcat2022-06-14https://gitlab.torproject.org/tpo/core/tor/-/issues/40443Write tests for Prop#324 code2023-06-27T21:21:57ZMike PerryWrite tests for Prop#324 codeHere's a list of tests we know we need for Prop#324 code. Likely will expand:
* [x] Test the number of cells that go into the RTT calculation for off-by-one, etc
* [x] Better tests for onion service negotiation
* [x] RTT and BDP algorit...Here's a list of tests we know we need for Prop#324 code. Likely will expand:
* [x] Test the number of cells that go into the RTT calculation for off-by-one, etc
* [x] Better tests for onion service negotiation
* [x] RTT and BDP algorithm tests (ideally as input + result vectors, for arti comparison)
* [x] CC alg window update tests (again ideally as input + result vectors)
* [x] Consensus parameter effects (possibly also as test vectors for above? but some can be simpler)
* [x] Test properties of flow control (ensuring against duplicate/excessive XONs, rate limit changes and bounds, etc)Tor: 0.4.7.x-post-stableMike PerryMike Perryhttps://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/40345Update Go in 11.0 series to 1.172021-12-20T19:18:47ZMatthew FinkelUpdate Go in 11.0 series to 1.17Following on from #40344, we need to start testing Go 1.17 in the alpha series relatively soon. However, we shouldn't move immediately onto 1.17 because we should wait some time for bugfixes and we need to solve #28325.Following on from #40344, we need to start testing Go 1.17 in the alpha series relatively soon. However, we shouldn't move immediately onto 1.17 because we should wait some time for bugfixes and we need to solve #28325.Tor Browser 11.5Georg KoppenGeorg Koppenhttps://gitlab.torproject.org/tpo/network-health/sbws/-/issues/40096codespell seems to fail intermittently2021-07-01T10:32:35ZGeorg Koppencodespell seems to fail intermittently```
ERROR: InvocationError for command /builds/gk/sbws/.tox/codespell/bin/codespell sbws tests docs (exited with code 65)
```
See: https://gitlab.torproject.org/gk/sbws/-/jobs/26705```
ERROR: InvocationError for command /builds/gk/sbws/.tox/codespell/bin/codespell sbws tests docs (exited with code 65)
```
See: https://gitlab.torproject.org/gk/sbws/-/jobs/26705sbws: 1.2.x-finalGeorg KoppenGeorg Koppen