The Tor Project issueshttps://gitlab.torproject.org/groups/tpo/-/issues2023-11-14T16:43:40Zhttps://gitlab.torproject.org/tpo/core/arti/-/issues/966hsclient descriptor download due to IPT nacks2023-11-14T16:43:40ZIan Jacksoniwj@torproject.orghsclient descriptor download due to IPT nacksAs per https://gitlab.torproject.org/tpo/core/arti/-/issues/913#note_2914467, Arti's current HS client behaviour wrt the service descriptor is:
> Ie if it's not expired we assume it's still good to use, even if we are making repeated at...As per https://gitlab.torproject.org/tpo/core/arti/-/issues/913#note_2914467, Arti's current HS client behaviour wrt the service descriptor is:
> Ie if it's not expired we assume it's still good to use, even if we are making repeated attempts to connect to the HS - with that descriptor - and they're all failing.
I am informed on IRC that the confirmation there, that this is correct behaviour, was due to a misunderstanding. This behaviour is in fact wrong.
If Arti gets NACKs from the IPTs, it needs to re-download the descriptor. (The service may have been restarted and our descriptor may therefore be out of date.)
In more detail:
* At some threshold of NACKs from IPTs, Arti should re-download the descriptor.
* If it manages to obtain a different descriptor it should try the IPTs in the new descriptor instead.
* "different" and "new" is according to the revision counter in the descriptor.
It's not clear precisely how hard Arti should try to obtain a different descriptor. After all, the service may not have managed to update all the hsdirs. So maybe Arti should try several.
Details of the correct retry algorithm aren't clear. Ideally the overall system would have the following properties:
* Restarting an HS service can be done without generating a persistent loss of availability on clients
* Faulty hsdirs don't cause loss of availability provided there are enough working hsdirs
* Trouble from faulty relays or "bad weather" isn't "magnified" by the hsdir/descriptor system
*This* ticket represents the required changes to Arti's client behaviour. There are also implications for the work-in-progress server-side work, eg !1429
CC @nickm @dgouletArti: Feature parity with the C implementationIan Jacksoniwj@torproject.orgIan Jacksoniwj@torproject.orghttps://gitlab.torproject.org/tpo/core/tor/-/issues/40893New conflux links every 30 sec when unused connections2024-01-25T21:08:13ZcypherpunksNew conflux links every 30 sec when unused connectionsAbout 30? minutes without use when tor drops the last connection, it creates 6 new Conflux_linked connections that drops after 30 seconds. Then replaces them with a new set of 6 Conflux_linked only for 30 seconds. And continues this loo...About 30? minutes without use when tor drops the last connection, it creates 6 new Conflux_linked connections that drops after 30 seconds. Then replaces them with a new set of 6 Conflux_linked only for 30 seconds. And continues this loop for ever until normal tor use is resumed.
latest commit tested: cec6f9919d3128646d85c75d08338bea4b31bffa
linux 6.4
This behavior exist at least a couple of months, before the adoption of the 4.8 series from the tor browser.Tor: 0.4.8.x-post-stableMike PerryMike Perryhttps://gitlab.torproject.org/tpo/anti-censorship/lox/-/issues/28Set daily max bucket distribution and adjust other settings for production2024-02-15T16:52:09ZonyinyangSet daily max bucket distribution and adjust other settings for productionWe likely need to decide on an upper bound of buckets that can be distributed each day so that we don't run out of open invitation buckets. We currently have buckets being distributed to k users before a new bucket is used but if buckets...We likely need to decide on an upper bound of buckets that can be distributed each day so that we don't run out of open invitation buckets. We currently have buckets being distributed to k users before a new bucket is used but if buckets are continuously requested, we will eventually run out of buckets each day. These variables should be part of a configuration file for Lox.Lox Ready for Open Testing Callonyinyangonyinyanghttps://gitlab.torproject.org/tpo/core/tor/-/issues/40832Memory consumption of Tor client is becoming difficult under iOS 50 MB RAM li...2023-10-09T15:34:54ZtlaMemory consumption of Tor client is becoming difficult under iOS 50 MB RAM limitation esp. during startup and with cached info### Summary
Evidence is gathering, that Tor under iOS' network extension memory limitation has trouble building circuits.
See https://github.com/guardianproject/orbot-apple/issues/71#issuecomment-1666818716 and comments before.
I'm pl...### Summary
Evidence is gathering, that Tor under iOS' network extension memory limitation has trouble building circuits.
See https://github.com/guardianproject/orbot-apple/issues/71#issuecomment-1666818716 and comments before.
I'm playing around with `MaxMemInQueues`, but as soon as I uppen it just a little from the 5MB we currently use, Jetsam starts to kill the Network Extension during startup circuit building. (At least in my Austrian environment.)
Starting with cashed information seems to become more and more difficult, too. I already had to put a "clear cache" button at the main screen, because people started to complain so much.
I witness this too, now, more and more often.
Do you see any possibility to reduce non-file-backed memory consumption during the startup phase?
If this continues to worsen, it will render Orbot iOS unusable.
I'm not sure if this is happening due to changes in the network or due to changes in the client code.
If this doesn't get better I will need to consider downgrading Tor to older versions again.
Esp. since I currently released Onion Browser version 3, which now relies on Orbot iOS completely and which currently makes a lot of users unhappy.
There's certain loopholes to the memory accounting in iOS. In this older article I wrote, there's some background to the Jetsam memory accounting:
https://benjaminerhart.com/2018/03/state-of-the-onion-ios/
Esp. helpful might be these:
- Use file-backed memory / operate on/stream files directly instead of loading everything and juggling that data around a lot. (Maybe we can do that in callbacks we can implement with Objective-C to make use of `NSCache`/`NSPurgableData` objects.)
- Provide a method which makes Tor give up unused memory. We could call that from a hook iOS provides.
### Steps to reproduce:
1. Install Orbot iOS
2. Start Orbot
3. Witness Tor protocol using left top button.
4. If under censored environment, Tor might not be able to build usable circuits at all, because `MaxMemInQueues` is set to 5 MByte.
5. Override and increase `MaxMemInQueues` in advanced settings.
6. Witness Network Extension crash during start. (App will show stopped status after a while.)
7. In non-restraint environments, manage a complete start. Confirm by surfing to check.torproject.org.
8. Stop Orbot.
9. Restart Orbot.
10. Restart will fail most of the time until "Clear Cache" is pressed.
### What is the current bug behavior?
### What is the expected behavior?
- Tor limits memory usage during startup phase, so we can increase `MaxMemInQueues`, so constrained environments work, too.
- Tor limits memory usage during loading of cached information, so it doesn't reach the 50 MB memory limit.
### Environment
| | |
|:-------- | --------:|
| tor | 0.4.7.13 |
| libevent | 2.1.12 |
| OpenSSL | 1.1.1u |
| liblzma | 5.4.3 |
iOS 16.6. using [Tor.framework](https://github.com/iCepa/Tor.framework/)
### Relevant logs and/or screenshots
https://github.com/guardianproject/orbot-apple/issues/71#issuecomment-1657738391
https://github.com/guardianproject/orbot-apple/issues/71#issuecomment-1666818716
### Possible fixesTor: 0.4.9.x-freezeAlexander Færøyahf@torproject.orgAlexander Færøyahf@torproject.orghttps://gitlab.torproject.org/tpo/core/tor/-/issues/40791prop340: Implement packed and fragmented cells2023-09-19T18:12:28ZDavid Gouletdgoulet@torproject.orgprop340: Implement packed and fragmented cellsThis ticket is about implementing proposal 340 on the relay side of C-tor.
Important to note that we plan to only implement this support client side in arti.This ticket is about implementing proposal 340 on the relay side of C-tor.
Important to note that we plan to only implement this support client side in arti.Tor: 0.4.9.x-freezeDavid Gouletdgoulet@torproject.orgDavid Gouletdgoulet@torproject.orghttps://gitlab.torproject.org/tpo/core/tor/-/issues/11101Bridges should report implementation versions of their pluggable transports2024-03-05T15:17:58ZRoger DingledineBridges should report implementation versions of their pluggable transportsOur bridges now run a variety of pluggable transports. What if there's a bug in, say, the Scramblesuit implementation (like it appears there is)? If we fix the bug, how do bridgedb or the Tor clients know whether the Scramblesuit bridge ...Our bridges now run a variety of pluggable transports. What if there's a bug in, say, the Scramblesuit implementation (like it appears there is)? If we fix the bug, how do bridgedb or the Tor clients know whether the Scramblesuit bridge they just learned about is one of the new (updated) ones or one of the old (buggy) ones?
One option would be for Tor to include a version for each supported PT in its bridge (or extrainfo) descriptor, so if we turn out to not want to use certain versions for certain situations, we can do it.
Are there better options than this one?Tor: 0.4.9.x-freezeDavid Gouletdgoulet@torproject.orgDavid Gouletdgoulet@torproject.orghttps://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/42490Install svg from branding theme to browser/chrome/icons/default2024-03-28T17:24:40ZboklmInstall svg from branding theme to browser/chrome/icons/defaultIn
https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/merge_requests/943#note_3013584,
@PieroV mentioned that we have svg files (added in commits `Bug 2176:
Rebrand Firefox to TorBrowser` and `MB 1: Mullvad Browser brandi...In
https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/merge_requests/943#note_3013584,
@PieroV mentioned that we have svg files (added in commits `Bug 2176:
Rebrand Firefox to TorBrowser` and `MB 1: Mullvad Browser branding`),
but we currently don't install them to `browser/chrome/icons/default`.
If we do that we can then update the debian package to link them from
`/usr/share/icons/hicolor/scalable/apps`.Pier Angelo VendramePier Angelo Vendramehttps://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/42489Lox module notifications2024-03-28T10:07:16ZhenryLox module notificationsCurrently the Lox module does not give out notifications for when its internals change, so:
+ `TorSetting` is not notified when the bridges change. So it is only set to the new value when restarting Tor Browser.
+ "about:preferences" is...Currently the Lox module does not give out notifications for when its internals change, so:
+ `TorSetting` is not notified when the bridges change. So it is only set to the new value when restarting Tor Browser.
+ "about:preferences" is not notified when the invites change or when there is a blockage or upgrade event.henryhenryhttps://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/41112Fix indentation of projects/browser/RelativeLink/start-browser2024-03-27T16:54:03ZboklmFix indentation of projects/browser/RelativeLink/start-browserThe file `projects/browser/RelativeLink/start-browser` is currently
indented with a mix of tabs, 8, 4 or 2 spaces.
I think we can change it to 2 spaces everywhere.The file `projects/browser/RelativeLink/start-browser` is currently
indented with a mix of tabs, 8, 4 or 2 spaces.
I think we can change it to 2 spaces everywhere.boklmboklmhttps://gitlab.torproject.org/tpo/network-health/metrics/descriptorParser/-/issues/88Pick up metrics-sql-tables changes for recommended*version2024-03-27T08:42:48ZGeorg KoppenPick up metrics-sql-tables changes for recommended*versionWe changed the column for recommended*version (to recommended*versions) over in https://gitlab.torproject.org/tpo/network-health/metrics/metrics-sql-tables/-/issues/12. We need to pick that commit up and make the related changes in descr...We changed the column for recommended*version (to recommended*versions) over in https://gitlab.torproject.org/tpo/network-health/metrics/metrics-sql-tables/-/issues/12. We need to pick that commit up and make the related changes in descriptorparser code.Georg KoppenGeorg Koppenhttps://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/42480Use translation CI in android2024-03-26T17:22:26ZhenryUse translation CI in androidhttps://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/42305 added some CI for automatically moving strings from `tor-browser` (desktop) to the translation repository.
The CI and bot seem to be working well, so I think we c...https://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/42305 added some CI for automatically moving strings from `tor-browser` (desktop) to the translation repository.
The CI and bot seem to be working well, so I think we can do the same in android.
/cc @pierov @dan @clairehursthenryhenryhttps://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/42479Switch from localized strings to error codes in TorConnect errors2024-03-28T17:23:09ZPier Angelo VendrameSwitch from localized strings to error codes in TorConnect errorsAs noted in !938, we try to translate some error messages in `TorConnect`.
However, it isn't great, because:
1. it's backend, localized strings don't belong there
2. it's backend, we need to pass stuff usable by the frontends in code (...As noted in !938, we try to translate some error messages in `TorConnect`.
However, it isn't great, because:
1. it's backend, localized strings don't belong there
2. it's backend, we need to pass stuff usable by the frontends in code (especially important for Android), it's the frontend's role to transform it in something for the users
3. for Android, this part of the code lives in GeckoView, which we currently don't translatePier Angelo VendramePier Angelo Vendramehttps://gitlab.torproject.org/tpo/network-health/metrics/metrics-bin/-/issues/3Run cleanup and other checks for NS API build2024-03-26T07:25:52ZMattia RighettiRun cleanup and other checks for NS API buildIt could be useful to do some cleanup of the build folder each time we need to build a new version of the ns api
Referencing https://gitlab.torproject.org/tpo/network-health/metrics/networkstatusapi/-/issues/54#note_3011903It could be useful to do some cleanup of the build folder each time we need to build a new version of the ns api
Referencing https://gitlab.torproject.org/tpo/network-health/metrics/networkstatusapi/-/issues/54#note_3011903Mattia RighettiMattia Righettihttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40352Use unreliable and unordered WebRTC data channels2024-03-21T20:15:25ZDavid Fifielddcf@torproject.orgUse unreliable and unordered WebRTC data channels@shelikhoo:
Actually here are some observation from me related to https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40243:
Snowflake is currently using network resource in a so suboptimal way I ...@shelikhoo:
Actually here are some observation from me related to https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40243:
Snowflake is currently using network resource in a so suboptimal way I think it would make sense to also consider make protocol level change on how kcp is interacting with webrtc before considering to add forward error correction. This would be in the form of enabling unreliable mode of webrtc and make necessary change to get it to work.
Right now, kcp packets are sent in webrtc data channel in a reliable way that deliver all packets in order and retransmit any lost message repeatedly. However, kcp also retransmit its packet itself, which as a result, queue all those retransmitted packets somewhere, like in webrtc's buffer.
This means lost packets are required to be retransmitted more than once in different protocol, and retransmit & timeout get compounded. More retransmit result in more lost packets and more retransmission, which eventually lead to [connection melt down](https://openvpn.net/faq/what-is-tcp-meltdown/) <- please read.
back pressure like the one introduced in https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/merge_requests/144 only move the problem, and block kcp's send in unexpected way, as kcp don't expect send to block as it is usually over udp.
See also: https://lists.torproject.org/pipermail/anti-censorship-team/2023-March/000286.html
(@dcf split this issue off from #40251 to separate the analysis of speed in China from the proposed remedy.)shelikhooshelikhoohttps://gitlab.torproject.org/tpo/network-health/metrics/website/-/issues/40111metrics.tpo not producing 'userstats-bridge-country' and 'userstats-bridge-co...2024-03-27T09:41:41ZHirometrics.tpo not producing 'userstats-bridge-country' and 'userstats-bridge-combined' graphs since Mar 15@gus noticed we are not producing graphs for 'userstats-bridge-country' and 'userstats-bridge-combined' since Mar 15th. The data is in the db, but the csv served by R are not being written.@gus noticed we are not producing graphs for 'userstats-bridge-country' and 'userstats-bridge-combined' since Mar 15th. The data is in the db, but the csv served by R are not being written.HiroHirohttps://gitlab.torproject.org/tpo/network-health/sbws/-/issues/40197Figure out why stream bandwidth is below 11MB for all relays2024-03-25T11:27:38ZjugaFigure out why stream bandwidth is below 11MB for all relaysWorking on sbws#40190 @gk found that measuring nth relays with https://gitlab.torproject.org/juga/relay_bw, none of the chosen paths nor Web servers measure more than 11MB.
Querying the new metrics DB, i realized that this is also the ca...Working on sbws#40190 @gk found that measuring nth relays with https://gitlab.torproject.org/juga/relay_bw, none of the chosen paths nor Web servers measure more than 11MB.
Querying the new metrics DB, i realized that this is also the case for sbws.
Could this be due to some tor configuration or limitation that we haven't realized?
Working on tor_fusion#5 with hiro, i realized that https://gitlab.torproject.org/tpo/network-health/metrics/onionperf/-/blob/74074b0c61c68edf2c5a9f20d437ccf3d9c5125a/onionperf/visualization.py#L185 is calculating the throughput by taking only the time it takes to download the last MiB of data.
I observed in the past sbws download speed was quite irregular, not that it was getting faster over time, but i might be wrong.
We didn't finish implementing https://gitlab.torproject.org/tpo/network-health/onbasca/-/issues/128 to check the bandwidth after the slow start.jugajugahttps://gitlab.torproject.org/tpo/network-health/metrics/metrics-sql-tables/-/issues/11Speed up queries2024-03-19T08:35:47ZjugaSpeed up queriesWorking on tpo/network-health/team#313 i filled up tpo/tpa/team#41558 so that a query that takes 50secs can work in grafana, but we should still speed up the queries by creating indexes for example.
We can collect here the slowest queri...Working on tpo/network-health/team#313 i filled up tpo/tpa/team#41558 so that a query that takes 50secs can work in grafana, but we should still speed up the queries by creating indexes for example.
We can collect here the slowest queries and how we can improve them.jugajugahttps://gitlab.torproject.org/tpo/team/-/issues/269s144 report2024-03-19T19:40:45ZGabagaba@torproject.orgs144 report2024-03-25https://gitlab.torproject.org/tpo/team/-/issues/268S150 Report2024-03-26T20:36:33ZGabagaba@torproject.orgS150 Report2024-04-01https://gitlab.torproject.org/tpo/tpa/team/-/issues/41557order and setup new backup server (bungei 2 AKA bacula-storage-02)2024-03-28T12:44:38Zanarcatorder and setup new backup server (bungei 2 AKA bacula-storage-02)The new backup server (#41364) has been approved.
@lavamind can you settle the specs, and order the box? i'm happy to review and approve the hardware if you don't feel fully confident, of course...
i also put the server setup in this i...The new backup server (#41364) has been approved.
@lavamind can you settle the specs, and order the box? i'm happy to review and approve the hardware if you don't feel fully confident, of course...
i also put the server setup in this issue, but we can also spin that out in a separate one if we want to... one thing that's for sure is we want to move to barman for the psql backups (#40950), but one ... uh... problem with that approach is that we actually want cross-backups, so, technically, we *can't* actually deploy *that* on the new server ... oops?
so maybe we need to move other psql databases and we'll have to resize the partition anyways? urghl.
thanks!Jérôme Charaouilavamind@torproject.orgJérôme Charaouilavamind@torproject.org