The Tor Project issueshttps://gitlab.torproject.org/groups/tpo/-/issues2024-01-25T15:40:14Zhttps://gitlab.torproject.org/tpo/core/arti/-/issues/1207replaylog partial write bug2024-01-25T15:40:14ZIan Jacksoniwj@torproject.orgreplaylog partial write bugIn `ReplayLog::check_inner`
```
// TODO HSS if write_all fails, it might have written part of the data;
// in that case, we must truncate the file to resynchronise.
// We should probably set a note to...In `ReplayLog::check_inner`
```
// TODO HSS if write_all fails, it might have written part of the data;
// in that case, we must truncate the file to resynchronise.
// We should probably set a note to truncate just before we call write_all
// and clear it again afterwards.
```Arti: Onion service supportIan Jacksoniwj@torproject.orgIan Jacksoniwj@torproject.orghttps://gitlab.torproject.org/tpo/core/arti/-/issues/1205code duplication between FsStateMgr and state_dir (!1853)2024-01-24T19:03:05ZIan Jacksoniwj@torproject.orgcode duplication between FsStateMgr and state_dir (!1853)Hi, @nickm, you asked me to write a ticket about state_dir vs tor_persist::FsStageMgr.
I think the situation is like this:
Background, contents of the state_dir structs: `state_dir::InstanceStateHandle` will have a `CheckedDir` for the...Hi, @nickm, you asked me to write a ticket about state_dir vs tor_persist::FsStageMgr.
I think the situation is like this:
Background, contents of the state_dir structs: `state_dir::InstanceStateHandle` will have a `CheckedDir` for the instance-specific state directory, and a lock guard (the lock guard is in the sketch `state_dir.rs`; the `CheckedDir` isn't but one couldn't implement it wihtout). `state_dir::StorageHandle` needs to contain a representation of the path, maybe a mistrust, and another clone of the lock guard, in order to implement the store/load/delete methods.
The actual implementation of the store/load/delete methods needs to be quite similar to those in `tor_persist/src/fs.rs`. (`FsStateMgr::load` and `store`)
But `state_dir` can't call that code because it's entangled with `FsStateMgr` which has its own locking arrangements. (I'm pretty sure `state_dir` doesn't want to try to reuse `FsStateMgr`s locking arrangements, since state_dir has different semantics.)
Looking at the amount of code invovled, it's 40loc including all the generics etc., so maybe just c&p it is OK. It doesn't fill me with happiness.
The obvious answer is to move the implementation of `FsStageMgr::load` and `store` into free functions that take an `&CheckedDir` or a `&Path`, and rely on the caller to do any locking. Then state_dir could just call those.
I'm not sure we want to expose those as a public API of tor-persist though. Maybe instead, we should move state_dir to tor-persist before we implement it, rather than trying to keep it in tor-hsservice. That would make this trivial.
In terms of what I'd like from you: I think the actual code movement or whatever here is straightforward. What I think is needed is a decision about what approach to take.Nick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/arti/-/issues/1196Remove all vestiges of HsClientIntroAuthKeypair and friends2024-01-10T20:11:25ZNick MathewsonRemove all vestiges of HsClientIntroAuthKeypair and friendsThese are deprecated, and we can probably afford to remove them soon.These are deprecated, and we can probably afford to remove them soon.https://gitlab.torproject.org/tpo/core/arti/-/issues/1192Pick a name for the subcomponents of an ArtiPathComponent2024-02-07T14:48:50Zgabi-250Pick a name for the subcomponents of an ArtiPathComponent#### Context
We need a name for the individual subcomponents (substrings) of non-leaf `ArtiPathComponent`s. These subcomponents (let's call them "slugs" for the purposes of this ticket) have the following properties:
* slugs can be co...#### Context
We need a name for the individual subcomponents (substrings) of non-leaf `ArtiPathComponent`s. These subcomponents (let's call them "slugs" for the purposes of this ticket) have the following properties:
* slugs can be concatenated to build file names
* when concatenating slugs, they should be separated using `/`, `+`, or `.`. The first slug should not be empty
* slugs should not be concatenated without separators (for security reasons)
* their charset is: lowercase ASCII alphanumerics and underscore. We may extend this to allow additional characters in the future, but `/`, `+`, and `.` (the slug separators) will never be valid slug characters
* they may be empty, but most (all?) of our use cases don't allow empty slugs
`HsNickname`s, non-leaf `ArtiPathComponent`s are slugs, as are the non-leaf components that make up `KeySpecifier`. The leaf component of an `ArtiPath` consists of one or more slugs, concatenated using `+` (for example, some keys have a `TimePeriod` slug)
#### Some possible names
Here are some possible names for the slugs described here:
* Slug
* PathElement
* FnameElement
* NameElement
* IdElt
* PathSlug
* IdStr
* FsId
* FnameId
* PathBlob
* PathSlug
* PathChunk
* Name
* Nick
* ...something else?
#### Platform-specific restrictions
On Windows, the following slugs are forbidden:
* con
* prn
* aux
* nul
* com1, com2, com3, com4, com5, com6, com7, com8, com9, com0
* lpt1, lpt2, lpt3, lpt4, lpt5, lpt6, lpt7, lpt8, lpt9, lpt0Arti: Onion service supporthttps://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/42362"New window" missing from File menu on macOS2024-01-10T07:30:30Zdonuts"New window" missing from File menu on macOSAt some point we seem to have lost the option to open a new window via the File menu on macOS.
[macos-file-menu](/uploads/aa06119db881fb40797f7500c3057e1c/macos-file-menu.png)At some point we seem to have lost the option to open a new window via the File menu on macOS.
[macos-file-menu](/uploads/aa06119db881fb40797f7500c3057e1c/macos-file-menu.png)https://gitlab.torproject.org/tpo/core/arti/-/issues/1179Upgrade from async-rustls to futures-rustls2024-01-25T16:03:50ZNick MathewsonUpgrade from async-rustls to futures-rustlsAccording to the async-rustls crate, it is now deprecated, and we should use futures-rustls instead.According to the async-rustls crate, it is now deprecated, and we should use futures-rustls instead.Nick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/42355Fullscreen on Android doesn't hide system bars2024-01-11T16:31:14ZPier Angelo VendrameFullscreen on Android doesn't hide system barsIn Tor Browser 13.0.7, when you go on fullscreen on Android, system bars persist.
So, as a matter of fact, it isn't a real fullscreen.
STR:
1. Open a video (tried on Invidious instances, but also other sites)
2. Go to fullscreen
3. No...In Tor Browser 13.0.7, when you go on fullscreen on Android, system bars persist.
So, as a matter of fact, it isn't a real fullscreen.
STR:
1. Open a video (tried on Invidious instances, but also other sites)
2. Go to fullscreen
3. Notice that the both the bar with clock/notifications and the "navigation" bar are still visible (I'm using gesture-based navigation, so I don't have a bar with buttons, but a bar with a horizontal line)
If you open the same site on Firefox (but without a custom config such as ours), fullscreen works as expected.
It might have started after the security backport.
Tested on my Pixel 4a, I've seen the problem both on 13.0.7 and on 13.5a3.
/cc @ma1ma1ma1https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/webtunnel/-/issues/30Best practices for update webtunnel in production2024-02-23T14:58:41ZJacobo NájeraBest practices for update webtunnel in productionHi,
What do you recommend to update Webtunnel when you use docker-compose setup?
From my side, it doesn't work
- docker compose pull
- docker compose up --force-recreate --build -d
I'm using "force-recreate" because the latest webt...Hi,
What do you recommend to update Webtunnel when you use docker-compose setup?
From my side, it doesn't work
- docker compose pull
- docker compose up --force-recreate --build -d
I'm using "force-recreate" because the latest webtunnel image is unchanged. But there is a new version of Tor. But it maintains Tor 0.4.7.13 version.
I also tried with it, but it doesn't work to update Tor version:
- docker compose down --volumes
- docker compose pull
- docker compose build
- docker compose up -d
Thanks, Jacoboshelikhooshelikhoohttps://gitlab.torproject.org/tpo/applications/mullvad-browser/-/issues/259"Your default search engine is not respectful of your privacy"2024-03-27T20:34:29Ztwo"Your default search engine is not respectful of your privacy"1. click the mullvad icon on the top
2. click the wheel icon
3. scroll to bottom
> Your default search engine is not respectful of your privacy. We recommend you switch to a more private one.
the page in "Learn More" suggests to switch...1. click the mullvad icon on the top
2. click the wheel icon
3. scroll to bottom
> Your default search engine is not respectful of your privacy. We recommend you switch to a more private one.
the page in "Learn More" suggests to switch to duckduckgo, but i already have duckduckgo (it's default)https://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/42347Add a banner warning users about the upcoming EOL for Win ≤8.1 and macOS ≤10.142024-02-01T14:39:44ZPier Angelo VendrameAdd a banner warning users about the upcoming EOL for Win ≤8.1 and macOS ≤10.1413.5 will be the last Windows version, Mozilla bumped the requirement to Windows 10.
We could add a warning to Windows 7 users somewhere, e.g., in about:tor.
We can check the version of the OS with `Services.sysinfo.getProperty("versio...13.5 will be the last Windows version, Mozilla bumped the requirement to Windows 10.
We could add a warning to Windows 7 users somewhere, e.g., in about:tor.
We can check the version of the OS with `Services.sysinfo.getProperty("version")`. It's `10.0` in my Windows 10 VM, and `6.1` in my Windows 7 VM.
Maybe the first versions of Windows 10 also use 6.1, but if that's the case, they're unsupported too.https://gitlab.torproject.org/tpo/community/team/-/issues/96Call for testers - TBA Alpha Connect Assist2024-03-05T19:13:47ZGusCall for testers - TBA Alpha Connect AssistApplications Team is implementing "connect assist" on TBA and it would be nice to collect users feedback. We should create a Call for Testers on the Forum.
Connect Assist is available on 13.5a3 alpha.
```
## Connect assist comes to And...Applications Team is implementing "connect assist" on TBA and it would be nice to collect users feedback. We should create a Call for Testers on the Forum.
Connect Assist is available on 13.5a3 alpha.
```
## Connect assist comes to Android
This alpha is the first release with Desktop's censorship circumvention feature (connect assist) available on Android! You can try it out for yourself by navigating to the `Settings > Tor Network` and selecting `Enable beta connection features`.
**NOTE**: This feature is very much in development and is still quiet brittle and liable to breakage. Do *not* toggle this option if you value your current Tor Browser for Android Alpha configuration. If this option gets the app into an unusable state, you can get back to the legacy bootstrapping system by clearing the app's storage and cache via the Android system settings. Of course, doing so will also delete any existing configuration options so please do not enable this option if this would be a problem for you.
When this feature ships in 13.5 in late spring/early summer 2024, Android users connecting from censored networks should have the exact same bootstrapping functionality Desktop users currently have. If they are unable to connect to the Tor Network, Tor Browser will query the anti-censorship backend to determine which pluggable transports and bridges they need to use based upon their locale and the current censorship landscape. The browser will then attempt to bootstrap once more with the new settings applied. Our hope is that this will take a lot of the guesswork out of connecting to the Tor Network for our mobile users, and ensure unfettered network access during censorship events.
For now we expect this new bootstrapping UX to be buggy, but we wanted to get it into the hands of users as early as possible so we can iterate and find and fix bugs as early as possible.
```Sponsor 96: Rapid Expansion of Access to the Uncensored Internet through Tor in China, Hong Kong, & TibetGusGus2024-03-01https://gitlab.torproject.org/tpo/tpa/team/-/issues/41454Migrate metrics-store-01 to object storage2024-01-04T19:34:23ZHiroMigrate metrics-store-01 to object storageWe have agreed we can migrate metrics-store-01 to object storage.We have agreed we can migrate metrics-store-01 to object storage.HiroHirohttps://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/42344Dragging and dropping files on Tor Browser triggers an assertion2024-01-08T11:13:11ZPier Angelo VendrameDragging and dropping files on Tor Browser triggers an assertionDropping a file on Tor Browser to open it triggers an assertion when they are enabled, on the validity of the event timestamp.
I haven't checked if Firefox also has this problem yet, because I don't have a debug build of Firefox.
Verif...Dropping a file on Tor Browser to open it triggers an assertion when they are enabled, on the validity of the event timestamp.
I haven't checked if Firefox also has this problem yet, because I don't have a debug build of Firefox.
Verified on Linux, for now, but might happen also on other platforms.
```c++
TimeDuration operator-(const TimeStamp& aOther) const {
MOZ_ASSERT(!IsNull(), "Cannot compute with a null value");
MOZ_ASSERT(!aOther.IsNull(), "Cannot compute with aOther null value"); // <-- this is the assertion
```
<details><summary>Stack trace</summary>
```
mozilla::TimeStamp::operator-(mozilla::TimeStamp const&) const (/home/piero/Tor/tor-browser/obj-x86_64-pc-linux-gnu/dist/include/mozilla/TimeStamp.h:478)
mozilla::TimeStamp mozilla::SystemTimeConverter<unsigned int, mozilla::TimeStamp>::GetTimeStampFromSystemTime<mozilla::CurrentX11TimeGetter>(unsigned int, mozilla::CurrentX11TimeGetter&) (/home/piero/Tor/tor-browser/widget/SystemTimeConverter.h:110)
nsWindow::GetEventTimeStamp(unsigned int) (/home/piero/Tor/tor-browser/widget/gtk/nsWindow.cpp:4925)
nsWindow::GetWidgetEventTime(unsigned int) (/home/piero/Tor/tor-browser/widget/gtk/nsWindow.cpp:4893)
nsWindow::OnEnterNotifyEvent(_GdkEventCrossing*) (/home/piero/Tor/tor-browser/widget/gtk/nsWindow.cpp:4237)
enter_notify_event_cb(_GtkWidget*, _GdkEventCrossing*) (/home/piero/Tor/tor-browser/widget/gtk/nsWindow.cpp:8023)
___lldb_unnamed_symbol9265 (@___lldb_unnamed_symbol9265:49)
___lldb_unnamed_symbol908 (@___lldb_unnamed_symbol908:104)
___lldb_unnamed_symbol1115 (@___lldb_unnamed_symbol1115:160)
g_signal_emit_valist (@g_signal_emit_valist:21)
g_signal_emit (@g_signal_emit:29)
___lldb_unnamed_symbol17566 (@___lldb_unnamed_symbol17566:109)
gtk_main_do_event (@gtk_main_do_event:474)
___lldb_unnamed_symbol2629 (@___lldb_unnamed_symbol2629:15)
___lldb_unnamed_symbol4021 (@___lldb_unnamed_symbol4021:14)
___lldb_unnamed_symbol2528 (@___lldb_unnamed_symbol2528:128)
___lldb_unnamed_symbol2539 (@___lldb_unnamed_symbol2539:151)
g_main_context_iteration (@g_main_context_iteration:18)
nsAppShell::ProcessNextNativeEvent(bool) (/home/piero/Tor/tor-browser/widget/gtk/nsAppShell.cpp:422)
nsBaseAppShell::DoProcessNextNativeEvent(bool) (/home/piero/Tor/tor-browser/widget/nsBaseAppShell.cpp:131)
nsBaseAppShell::OnProcessNextEvent(nsIThreadInternal*, bool) (/home/piero/Tor/tor-browser/widget/nsBaseAppShell.cpp:250)
non-virtual thunk to nsBaseAppShell::OnProcessNextEvent(nsIThreadInternal*, bool) (/home/piero/Tor/tor-browser/widget/nsBaseAppShell.cpp:0)
nsThread::ProcessNextEvent(bool, bool*) (/home/piero/Tor/tor-browser/xpcom/threads/nsThread.cpp:1154)
NS_ProcessNextEvent(nsIThread*, bool) (/home/piero/Tor/tor-browser/xpcom/threads/nsThreadUtils.cpp:479)
mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate*) (/home/piero/Tor/tor-browser/ipc/glue/MessagePump.cpp:85)
MessageLoop::RunHandler() (/home/piero/Tor/tor-browser/ipc/chromium/src/base/message_loop.cc:361)
MessageLoop::Run() (/home/piero/Tor/tor-browser/ipc/chromium/src/base/message_loop.cc:343)
nsBaseAppShell::Run() (/home/piero/Tor/tor-browser/widget/nsBaseAppShell.cpp:148)
nsAppStartup::Run() (/home/piero/Tor/tor-browser/toolkit/components/startup/nsAppStartup.cpp:295)
XREMain::XRE_mainRun() (/home/piero/Tor/tor-browser/toolkit/xre/nsAppRunner.cpp:5826)
XREMain::XRE_main(int, char**, mozilla::BootstrapConfig const&) (/home/piero/Tor/tor-browser/toolkit/xre/nsAppRunner.cpp:6026)
XRE_main(int, char**, mozilla::BootstrapConfig const&) (/home/piero/Tor/tor-browser/toolkit/xre/nsAppRunner.cpp:6082)
do_main(int, char**, char**) (/home/piero/Tor/tor-browser/browser/app/nsBrowserApp.cpp:227)
main (/home/piero/Tor/tor-browser/browser/app/nsBrowserApp.cpp:445)
__libc_start_call_main (@__libc_start_call_main:26)
__libc_start_main_impl (@__libc_start_main@@GLIBC_2.34:43)
_start (@_start:14)
```
</details>
Other kind of drag and drops (e.g., tabs/links from Firefox) work, but I haven't done an extended test.ma1ma1https://gitlab.torproject.org/tpo/network-health/metrics/descriptorParser/-/issues/69`unmeasured`, `vote` (and `consensus_bandwidth_is_unmeasured`) keywords are n...2024-01-16T13:49:09ZGeorg Koppen`unmeasured`, `vote` (and `consensus_bandwidth_is_unmeasured`) keywords are not properly treatedHave a look at the following 2 bw lines and there representation in our DB:
```
./16/2023-12-16-12-38-04-bandwidth-95BC922FA3DB2DDFC7E98ED4A9FFD0138A17BB1DEAD638852C601E55BD5878CF:bw=1 error_circ=5 error_destination=0 error_misc=0 error_...Have a look at the following 2 bw lines and there representation in our DB:
```
./16/2023-12-16-12-38-04-bandwidth-95BC922FA3DB2DDFC7E98ED4A9FFD0138A17BB1DEAD638852C601E55BD5878CF:bw=1 error_circ=5 error_destination=0 error_misc=0 error_second_relay=0 error_stream=0 master_key_ed25519=8VJa1wrFEuyszRnbE9z3D7acLWLvMrRgqodyrZzF1XM nick=TORPi node_id=FFDC0E2EA1D56C555978920C6D26EBA6AFF248FE relay_in_recent_consensus_count=103 relay_recent_measurement_attempt_count=5 relay_recent_measurements_excluded_error_count=5 relay_recent_priority_list_count=5 success=0 time=2023-12-16T00:41:02 unmeasured=1 vote=0 xoff_recv=0 xoff_sent=0
digest | bw | bw_mean | bw_median | consensus_bandwidth | consensus_bandwidth_is_unmeasured | desc_bw_avg | desc_bw_bur | desc_bw_obs_last | desc_bw_obs_mean | error_circ | error_destination | error_misc | error_second_relay | error_stream | master_key_ed25519 | nick | node_id | rtt | relay_in_recent_consensus_count | relay_recent_measurement_attempt_count | relay_recent_measurements_excluded_error_count | relay_recent_measurement_failure_count | relay_recent_measurements_excluded_near_count | relay_recent_measurements_excluded_old_count | relay_recent_measurements_excluded_few_count | relay_recent_priority_list_count | under_min_report | unmeasured | vote | xoff_recv | xoff_sent | success | time | bandwidth_file | r_strm | r_strm_filt
---------------------------------------------+----+---------+-----------+---------------------+-----------------------------------+-------------+-------------+------------------+------------------+------------+-------------------+------------+--------------------+--------------+---------------------------------------------+-------+------------------------------------------+-----+---------------------------------+----------------------------------------+------------------------------------------------+----------------------------------------+-----------------------------------------------+----------------------------------------------+----------------------------------------------+----------------------------------+------------------+------------+------+-----------+-----------+---------+---------------------+---------------------------------------------+--------+-------------
ehnR7l7BXiy24GF9TaeNYXjZBzWPCYff0VmoLRJL22I | 1 | -1 | -1 | -1 | f | -1 | -1 | -1 | -1 | 5 | 0 | 0 | 0 | 0 | 8VJa1wrFEuyszRnbE9z3D7acLWLvMrRgqodyrZzF1XM | TORPi | FFDC0E2EA1D56C555978920C6D26EBA6AFF248FE | 0 | 103 | 5 | 5 | 0 | 0 | 0 | 0 | 5 | f | f | f | 0 | 0 | 0 | 2023-12-16 00:41:02 | lbySL6PbLd/H6Y7Uqf/QE4oXux3q1jiFLGAeVb1YeM8 | 0 | 0
```
and
```
./16/2023-12-16-12-36-10-bandwidth-8AB15BA0804A1CB8B8D251523118156C49107E7DA4B1888965B196365A309864:bw=33000 bw_mean=1157384 bw_median=991611 consensus_bandwidth=34000000 consensus_bandwidth_is_unmeasured=False desc_bw_avg=1073741824 desc_bw_bur=1073741824 desc_bw_obs_last=25973760 desc_bw_obs_mean=30132316 error_circ=0 error_destination=0 error_misc=0 error_second_relay=0 error_stream=0 master_key_ed25519=zak6whupYxRKeYPtIfYJBlQrksWKIVeUYXuyc+WZBM4 nick=Najdorf node_id=$FAF0A8829E39063669FA609B904E0FB8D5E1F23F relay_in_recent_consensus_count=120 relay_recent_measurement_attempt_count=15 relay_recent_priority_list_count=15 success=15 time=2023-12-16T07:41:31
digest | bw | bw_mean | bw_median | consensus_bandwidth | consensus_bandwidth_is_unmeasured | desc_bw_avg | desc_bw_bur | desc_bw_obs_last | desc_bw_obs_mean | error_circ | error_destination | error_misc | error_second_relay | error_stream | master_key_ed25519 | nick | node_id | rtt | relay_in_recent_consensus_count | relay_recent_measurement_attempt_count | relay_recent_measurements_excluded_error_count | relay_recent_measurement_failure_count | relay_recent_measurements_excluded_near_count | relay_recent_measurements_excluded_old_count | relay_recent_measurements_excluded_few_count | relay_recent_priority_list_count | under_min_report | unmeasured | vote | xoff_recv | xoff_sent | success | time | bandwidth_file | r_strm | r_strm_filt
---------------------------------------------+-------+---------+-----------+---------------------+-----------------------------------+-------------+-------------+------------------+------------------+------------+-------------------+------------+--------------------+--------------+---------------------------------------------+---------+-------------------------------------------+-----+---------------------------------+----------------------------------------+------------------------------------------------+----------------------------------------+-----------------------------------------------+----------------------------------------------+----------------------------------------------+----------------------------------+------------------+------------+------+-----------+-----------+---------+---------------------+---------------------------------------------+--------+-------------
NzLgVvhWGgCgxBAaWw/XgeLQxHe3JT1OcZ2ChZWBHSU | 33000 | 1157384 | 991611 | 34000000 | f | 1073741824 | 1073741824 | 25973760 | 30132316 | 0 | 0 | 0 | 0 | 0 | zak6whupYxRKeYPtIfYJBlQrksWKIVeUYXuyc+WZBM4 | Najdorf | $FAF0A8829E39063669FA609B904E0FB8D5E1F23F | 0 | 120 | 15 | 0 | 0 | 0 | 0 | 0 | 15 | f | f | f | 0 | 0 | 15 | 2023-12-16 07:41:31 | irFboIBKHLi40lFSMRgVbEkQfn2ksYiJZbGWNlowmGQ | 0 | 0
```
The first one has `unmeasured=1` and `vote=0` while the latter has both of them missing. Yet in the DB both entries are `f`.
I guess in the first case the `unmeasured` should be `t` and `vote` should be `f` and in the second one it should be `f` and `t`, respectively?
/cc @jugahttps://gitlab.torproject.org/tpo/core/tor/-/issues/40901Document for the Relay Operator community how to debug relays that are slower...2023-12-19T07:53:56ZAlexander Færøyahf@torproject.orgDocument for the Relay Operator community how to debug relays that are slower than what the operator expectsThis idea origins from a conversation betweeh @beth, @gk and I on #tor-dev today.
We often release new features of C Tor to the relay operators that causes discussions/conversations around whether Tor has gotten faster/slower/uses (more...This idea origins from a conversation betweeh @beth, @gk and I on #tor-dev today.
We often release new features of C Tor to the relay operators that causes discussions/conversations around whether Tor has gotten faster/slower/uses (more|less) memory/crashes (more|less) often/etc. many of these items are hard to give a definitive "yes, the cause of this is X" and it's very time consuming for the Network Team to debug each item individually with the operator.
It would be very useful to have a document in place that informs relay operators about the different situations that may impact performance and how they can get some performance measurements they can then compare to and see if our performance have truly regressed. This can also be used to push MetricsPort to more operators.
We can expand upon the document over time as we discover new ways to do this analysis and/or from feedback from the relay operator community.
This is related to:
- https://lists.torproject.org/pipermail/tor-relays/2023-December/021409.html
- https://lists.torproject.org/pipermail/tor-relays/2023-December/021407.html
This may be relevant to Arti Relay too.
CC @mikeperry for awareness.https://gitlab.torproject.org/tpo/network-health/metrics/descriptorParser/-/issues/67Avoid NullPointerException when trying to get an exception message2024-01-16T13:49:11ZGeorg KoppenAvoid NullPointerException when trying to get an exception messageWile going over our parser logs I found this interesting bit:
```
2023-12-16 11:50:51,346 WARN o.t.m.d.u.DateTimeHelper:149 VoteParser run().
java.lang.NullPointerException: Cannot invoke "String.length()" because "s" is null
at java.ba...Wile going over our parser logs I found this interesting bit:
```
2023-12-16 11:50:51,346 WARN o.t.m.d.u.DateTimeHelper:149 VoteParser run().
java.lang.NullPointerException: Cannot invoke "String.length()" because "s" is null
at java.base/java.util.Formatter.parse(Formatter.java:2717)
at java.base/java.util.Formatter.format(Formatter.java:2671)
at java.base/java.util.Formatter.format(Formatter.java:2625)
at java.base/java.lang.String.format(String.java:4145)
at org.torproject.metrics.descriptorparser.parsers.VoteParser.addNetworkStatusVote(VoteParser.java:226)
at org.torproject.metrics.descriptorparser.parsers.VoteParser.run(VoteParser.java:85)
at org.torproject.metrics.descriptorparser.Main.run(Main.java:147)
at org.torproject.metrics.descriptorparser.Main.exec(Main.java:36)
at org.torproject.metrics.descriptorparser.Main.main(Main.java:32)
```
I guess what that means is we should be a bit more careful with blocks like
```
} catch (Exception ex) {
logger.warn("Exception. {}".format(ex.getMessage()));
ex.printStackTrace();
}
```
making sure we only try to get a message if it is not `null` (`NullPointerException`s have no messages, for instance, IIRC).https://gitlab.torproject.org/tpo/tpa/team/-/issues/41450Move collector.torproject.org to serve files stored in object storage2024-01-04T19:33:19ZHiroMove collector.torproject.org to serve files stored in object storageIn https://gitlab.torproject.org/tpo/tpa/team/-/issues/41416 we have discussed how we can move the tarballs from metrics-store-01 and those collector creates to object storage.
For metrics-store-01 we can just move the files, and once w...In https://gitlab.torproject.org/tpo/tpa/team/-/issues/41416 we have discussed how we can move the tarballs from metrics-store-01 and those collector creates to object storage.
For metrics-store-01 we can just move the files, and once we have the bucket, we can just update the links in the wiki where we list our archives.
For collector we need a way for people to browse the archives and download tarballs recursively if needed. I am thinking that we should preserve what we serve on collector.tpo, just have the links point to the buckets.
Once this is done, we can also discuss how we could generate the tarballs and move them to minio.https://gitlab.torproject.org/tpo/core/arti/-/issues/1167Implement a new revision-counter algorithm2024-01-23T17:22:20Zgabi-250Implement a new revision-counter algorithmIf we decide the answer to #1166 is that we should indeed continue including the next TP in the secondary ring, then this [discussion](https://gitlab.torproject.org/tpo/core/arti/-/merge_requests/1828#note_2976777) will need to be addres...If we decide the answer to #1166 is that we should indeed continue including the next TP in the secondary ring, then this [discussion](https://gitlab.torproject.org/tpo/core/arti/-/merge_requests/1828#note_2976777) will need to be addressed.
@nickm suggests
```
Okay. What's the right way for us to avoid that? I see a few options:
Just don't upload before the start of the time period.
Adjust our zero so that it is the start of the previous time period (or something like that), and increment the other values accordingly. We might need to increase the maximum on our OPE.
Kludge our OPE so that we can encrypt -N through 0 and usually get results in the same area as our C code.
Option 1 would presumably make us less reliable. (How far in advance do we upload, though? And are we right to do so?)
Option 2 would make our OPE less compatible with the C OPE. I think that's okay, though.
Option 3 would take some design, but I think I could figure something out.
```Arti: Onion service supporthttps://gitlab.torproject.org/tpo/core/arti/-/issues/1162Allow configuring other paths with respect to state_dir.2024-01-09T19:34:42ZNick MathewsonAllow configuring other paths with respect to state_dir.If you override `state_dir`, you might think that you've done enough to avoid using the default state directory in `${ARTI_LOCAL_DATA}` ... but you'd be wrong! The keystore location also defaults to `${ARTI_LOCAL_DATA}/keystore`, so you'...If you override `state_dir`, you might think that you've done enough to avoid using the default state directory in `${ARTI_LOCAL_DATA}` ... but you'd be wrong! The keystore location also defaults to `${ARTI_LOCAL_DATA}/keystore`, so you'd need to override that too.
It would be neat if we could define a directory as `${state_dir}/keystore`. But in order to do that, we would need to have the ability to let some `CfgPath`-like types get defined with respect to other `CfgPath`s. We would need some kind of a multi-step path resolution process.
To keep this simple, I'd suggest that we declare that there is only one level of dependencies here: that other paths may be defined in terms of `${state_dir}`, but that we don't allow every possible path to depend on every other possible path.
Is this worth doing?Arti: Onion service supporthttps://gitlab.torproject.org/tpo/core/arti/-/issues/1161"Missing previous key" warnings on restarting onion service2024-02-22T14:52:08ZNick Mathewson"Missing previous key" warnings on restarting onion serviceSometimes (but not always) when I restart an onion service, I get a bunch of warnings like this:
```
2023-12-13T13:25:58Z ERROR tor_hsservice::ipt_mgr: HS service hewwo missing previous key ArtiPath("hs/hewwo/ipts/k_hss_ntor+398509d88033...Sometimes (but not always) when I restart an onion service, I get a bunch of warnings like this:
```
2023-12-13T13:25:58Z ERROR tor_hsservice::ipt_mgr: HS service hewwo missing previous key ArtiPath("hs/hewwo/ipts/k_hss_ntor+398509d88033d16af8ef891fd3970d6f189e4a215ea6b40ebab6ea080c29dad4"), regenerating
2023-12-13T13:25:58Z ERROR tor_hsservice::ipt_mgr: HS service hewwo missing previous key ArtiPath("hs/hewwo/ipts/k_sid+398509d88033d16af8ef891fd3970d6f189e4a215ea6b40ebab6ea080c29dad4"), regenerating
2023-12-13T13:25:58Z ERROR tor_hsservice::ipt_mgr: HS service hewwo missing previous key ArtiPath("hs/hewwo/ipts/k_hss_ntor+44b23ce9420e67a94dd88c5529832adcaee6f8af227382e6d41daf643ca5ee82"), regenerating
2023-12-13T13:25:58Z ERROR tor_hsservice::ipt_mgr: HS service hewwo missing previous key ArtiPath("hs/hewwo/ipts/k_sid+44b23ce9420e67a94dd88c5529832adcaee6f8af227382e6d41daf643ca5ee82"), regenerating
2023-12-13T13:25:58Z ERROR tor_hsservice::ipt_mgr: HS service hewwo missing previous key ArtiPath("hs/hewwo/ipts/k_hss_ntor+3c8775e01e11ed6f6ea72a8057f425253d4584d7ff52c0487fbf88ffa32b18d3"), regenerating
2023-12-13T13:25:58Z ERROR tor_hsservice::ipt_mgr: HS service hewwo missing previous key ArtiPath("hs/hewwo/ipts/k_sid+3c8775e01e11ed6f6ea72a8057f425253d4584d7ff52c0487fbf88ffa32b18d3"), regenerating
```
There are two problems here:
First, these should not be at `ERROR`: We should only use ERROR for fatal problems.
Second, they really shouldn't be happening.
@diziet @gabi-250 Any idea what's up here?Ian Jacksoniwj@torproject.orgIan Jacksoniwj@torproject.org