Arti issueshttps://gitlab.torproject.org/tpo/core/arti/-/issues2024-03-28T17:26:55Zhttps://gitlab.torproject.org/tpo/core/arti/-/issues/1346Automatically detect broken links in our documentation?2024-03-28T17:26:55ZNick MathewsonAutomatically detect broken links in our documentation?Inspired by !2054, which fixed a broken link to spec.torproject.org.
Maybe it would be neat to have a tool to check out all the links in our documentation to make sure that they're all live and working. (This shouldn't be a CI blocker,...Inspired by !2054, which fixed a broken link to spec.torproject.org.
Maybe it would be neat to have a tool to check out all the links in our documentation to make sure that they're all live and working. (This shouldn't be a CI blocker, since websites can be unreliable.)https://gitlab.torproject.org/tpo/core/arti/-/issues/1345(Mostly) forbid futures::channel::oneshot and std::future::pending?2024-03-28T16:45:00ZNick Mathewson(Mostly) forbid futures::channel::oneshot and std::future::pending?See 95ed432c13c2c4b2d287f7a7a040576627687dbf and 62dd8e119940b851b01d2f6779c73cbdb4f430fb: we prefer `tor_async_utils::oneshot` and `future_pending`. Maybe we should enforce this to prevent bugs or issues.See 95ed432c13c2c4b2d287f7a7a040576627687dbf and 62dd8e119940b851b01d2f6779c73cbdb4f430fb: we prefer `tor_async_utils::oneshot` and `future_pending`. Maybe we should enforce this to prevent bugs or issues.https://gitlab.torproject.org/tpo/core/arti/-/issues/1344Add and use a count-enforcing hashmap for streammap.rs2024-03-28T14:28:51ZNick MathewsonAdd and use a count-enforcing hashmap for streammap.rsIn streammap.rs, after !2047, we keep a count of open streams. But the code to keep the count accurate is a little tricky; we might be better off having a hashmap that _only_ has the job of keeping the count accurate, and letting the re...In streammap.rs, after !2047, we keep a count of open streams. But the code to keep the count accurate is a little tricky; we might be better off having a hashmap that _only_ has the job of keeping the count accurate, and letting the rest of the streammap.rs code use that.Nick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/arti/-/issues/1342Use d-a for our ad-hoc BinaryHeap entry implementations2024-03-25T10:30:18Zgabi-250Use d-a for our ad-hoc BinaryHeap entry implementationsWe have a lot of types meant for use in a [std BinaryHeap](https://doc.rust-lang.org/std/collections/struct.BinaryHeap.html) to make it behave as a min-heap instead of max-heap. These types all have formulaic `Ord`/`ParialOrd`/`Eq`/`Part...We have a lot of types meant for use in a [std BinaryHeap](https://doc.rust-lang.org/std/collections/struct.BinaryHeap.html) to make it behave as a min-heap instead of max-heap. These types all have formulaic `Ord`/`ParialOrd`/`Eq`/`PartialEq` impls
```rust
impl Ord for ReuploadTimer {
fn cmp(&self, other: &Self) -> Ordering {
// Reversed, because we want the earlier
// `ReuploadTimer` to be "greater".
self.when.cmp(&other.when).reverse()
}
}
impl PartialOrd for ReuploadTimer {
fn partial_cmp(&self, other: &Self) -> Option<Ordering> {
Some(self.cmp(other))
}
}
impl PartialEq for ReuploadTimer {
fn eq(&self, other: &Self) -> bool {
self.when == other.when
}
}
impl Eq for ReuploadTimer {}
```
```rust
impl<TT: Ord, RD> Ord for RefetchEntry<TT, RD> {
fn cmp(&self, other: &Self) -> Ordering {
self.when.cmp(&other.when).reverse()
// We don't care about the ordering of BridgeConfig or retry_delay.
// Different BridgeConfig with the same fetch time will be fetched in "some order".
}
}
impl<TT: Ord, RD> PartialOrd for RefetchEntry<TT, RD> {
fn partial_cmp(&self, other: &Self) -> Option<Ordering> {
Some(self.cmp(other))
}
}
impl<TT: Ord, RD> PartialEq for RefetchEntry<TT, RD> {
fn eq(&self, other: &Self) -> bool {
self.cmp(other) == Ordering::Equal
}
}
impl<TT: Ord, RD> Eq for RefetchEntry<TT, RD> {}
```
```rust
impl PartialEq for SleepEntry {
fn eq(&self, other: &Self) -> bool {
self.when == other.when
}
}
impl Eq for SleepEntry {}
impl PartialOrd for SleepEntry {
fn partial_cmp(&self, other: &Self) -> Option<Ordering> {
Some(self.cmp(other))
}
}
impl Ord for SleepEntry {
fn cmp(&self, other: &Self) -> Ordering {
self.when.cmp(&other.when).reverse()
}
}
```
```rust
impl<TT: Ord, RD> Ord for RefetchEntry<TT, RD> {
fn cmp(&self, other: &Self) -> Ordering {
self.when.cmp(&other.when).reverse()
// We don't care about the ordering of BridgeConfig or retry_delay.
// Different BridgeConfig with the same fetch time will be fetched in "some order".
}
}
impl<TT: Ord, RD> PartialOrd for RefetchEntry<TT, RD> {
fn partial_cmp(&self, other: &Self) -> Option<Ordering> {
Some(self.cmp(other))
}
}
impl<TT: Ord, RD> PartialEq for RefetchEntry<TT, RD> {
fn eq(&self, other: &Self) -> bool {
self.cmp(other) == Ordering::Equal
}
}
impl<TT: Ord, RD> Eq for RefetchEntry<TT, RD> {}
```
And I'm about to add another such type!
ISTM these could all be auto-generated (probably with d-a).gabi-250gabi-250https://gitlab.torproject.org/tpo/core/arti/-/issues/1341Sync GeoIP Databases in an as automated way as possible2024-03-20T18:07:33ZAlexander Færøyahf@torproject.orgSync GeoIP Databases in an as automated way as possibleWith C Tor, @dgoulet made this nice toolchain where the CI generates the text files for us for GeoIP usage. It looks like we need something like this for Arti too as the GeoIP DB's are stale right now (not updated since d5632eacb2).With C Tor, @dgoulet made this nice toolchain where the CI generates the text files for us for GeoIP usage. It looks like we need something like this for Arti too as the GeoIP DB's are stale right now (not updated since d5632eacb2).Alexander Færøyahf@torproject.orgAlexander Færøyahf@torproject.orghttps://gitlab.torproject.org/tpo/core/arti/-/issues/1340Ensure we apply the necessary restrictions when building vanguard circuits2024-03-21T12:00:30Zgabi-250Ensure we apply the necessary restrictions when building vanguard circuitsThe following discussion from !2046 should be addressed:
- [ ] @nickm started a [discussion](https://gitlab.torproject.org/tpo/core/arti/-/merge_requests/2046#note_3010022):
> For both cases where you call select_vanguard, and for...The following discussion from !2046 should be addressed:
- [ ] @nickm started a [discussion](https://gitlab.torproject.org/tpo/core/arti/-/merge_requests/2046#note_3010022):
> For both cases where you call select_vanguard, and for when you use relay_selector below in this function:
>
> Watch out: just because we aren't enforcing family restrictions, doesn't mean we have no restrictions at all. In particular, I think we we have to prevent the same Relay from appearing twice consecutively. (A relay won't let you extend a circuit to itself.)
>
>
> @mikeperry Is that right? And is the only restriction on vanguard-based paths?
>
> This has implications for the select_vanguard API; maybe it needs to take a RelayExclusion.Arti: Guard discovery researchgabi-250gabi-250https://gitlab.torproject.org/tpo/core/arti/-/issues/1339Be consistent with our STUB/STUB+ terminology throughout circmgr/guardmgr2024-03-20T12:31:52Zgabi-250Be consistent with our STUB/STUB+ terminology throughout circmgr/guardmgrThe following discussion from !2046 should be addressed:
- [ ] @nickm started a [discussion](https://gitlab.torproject.org/tpo/core/arti/-/merge_requests/2046#note_3010014): (+3 comments)
> IMO we should do these things in the doc...The following discussion from !2046 should be addressed:
- [ ] @nickm started a [discussion](https://gitlab.torproject.org/tpo/core/arti/-/merge_requests/2046#note_3010014): (+3 comments)
> IMO we should do these things in the docs:
> - Explain what a stub circuit is
> - Explain when we would choose STUB and when we would choose STUB+
> - Get our terminology uniform on "STUB+" vs "Extended". (But let's not do a big rename until we've decided.)Arti: Guard discovery researchgabi-250gabi-250https://gitlab.torproject.org/tpo/core/arti/-/issues/1338Messages should be localisable (i18n)2024-03-19T14:22:53ZIan Jacksoniwj@torproject.orgMessages should be localisable (i18n)Currently we use English strings as (error) messages in many places, with no translation facility.
Tor is supposed to be a project for everyone, not just speakers of English. We should have a message translation system and use it throu...Currently we use English strings as (error) messages in many places, with no translation facility.
Tor is supposed to be a project for everyone, not just speakers of English. We should have a message translation system and use it throughout. (Except for internal errors, since the codebase is in English.)
Rust hasn't had a very good story about this, at least until recently. We should:
* Evaluate message translation systems for (at least) ease of integration, ease of calling in our codebase, and translator UX
* Consider whether we want to do anything better in the meantime for the error type variants that currently contain `&'static str`, which I currently feel dirty about every time.
We should likely consider other aspects of internationalisation too but that's not this ticket.https://gitlab.torproject.org/tpo/core/arti/-/issues/1337multiple toml configs (including cli options) array/map merge semantics2024-03-19T12:53:40ZIan Jacksoniwj@torproject.orgmultiple toml configs (including cli options) array/map merge semanticsTo give an example, suppose we have these two files:
```
# a.toml
[[entries]]
value = 42
# b.toml
[[entries]]
value = 666
```
This produces (using JSONish notation) `{ entries: [ { value: 42 }, { value: 666 } ] }`.
Now consider:
```...To give an example, suppose we have these two files:
```
# a.toml
[[entries]]
value = 42
# b.toml
[[entries]]
value = 666
```
This produces (using JSONish notation) `{ entries: [ { value: 42 }, { value: 666 } ] }`.
Now consider:
```
# a.toml
entries = [ { value = 42 } ]
# b.toml
entries = [ { value = 666 } ]
```
If we process both these files, in that order, semantically the result ought either to be "error" or equivalent to just `b.toml`. The latter is probably better since it allows overriding.
This situation can only be detected while the original TOML structure exists; it can't be handled correctly by a post-deserialisation merge.
Currently we don't document the actual behaviour of Arti. !2041 (and !1989) want to perhaps change the behaviour.
We should also see what other consumers of the Rust TOML libaries do. Notably, cargo merges config files sometimes. (My priors suggest that this aspect probably hasn't been properly considered in that context but I can hope to be pleasantly surprised.)
CC @nickmIan Jacksoniwj@torproject.orgIan Jacksoniwj@torproject.orghttps://gitlab.torproject.org/tpo/core/arti/-/issues/1336Optimize RelayCellBody::is_recognized2024-03-19T00:55:21ZJim NewsomeOptimize RelayCellBody::is_recognized!2045 adds some additional branching and complexity to `is_recognized` etc to handle different relay cell formats. This code is in the critical path for performance, so we may want to spend some more effort optimizing it.
It may also be...!2045 adds some additional branching and complexity to `is_recognized` etc to handle different relay cell formats. This code is in the critical path for performance, so we may want to spend some more effort optimizing it.
It may also be worth adding some (e.g. [criterion](https://github.com/bheisler/criterion.rs)) benchmarks for this code as well.https://gitlab.torproject.org/tpo/core/arti/-/issues/1335Build failure related to lseek64 on osx in build-repro ci2024-03-14T18:05:16ZNick MathewsonBuild failure related to lseek64 on osx in build-repro ciSee for example https://gitlab.torproject.org/tpo/core/arti/-/jobs/501742 :
```
/arti/osxcross/build/apple-libtapi/src/llvm/lib/Support/raw_ostream.cpp:834:11: error: no member named 'lseek64' in the global namespace; did you mean 'lsee...See for example https://gitlab.torproject.org/tpo/core/arti/-/jobs/501742 :
```
/arti/osxcross/build/apple-libtapi/src/llvm/lib/Support/raw_ostream.cpp:834:11: error: no member named 'lseek64' in the global namespace; did you mean 'lseek'?
834 | pos = ::lseek64(FD, off, SEEK_SET);
| ~~^~~~~~~
| lseek
/usr/include/unistd.h:46:7: note: 'lseek' declared here
46 | off_t lseek(int, off_t, int);
| ^
```https://gitlab.torproject.org/tpo/core/arti/-/issues/1334In tor-ptmgr, make managed pts optional2024-03-14T16:15:19ZNick MathewsonIn tor-ptmgr, make managed pts optionalWith !2043, tor-ptmgr will have support for unmanaged pts. But the current design means that anyone who only wants unmanaged pts will still have to carry the code for managed pts.
We might want to adjust `tor-ptmgr` so that managed plu...With !2043, tor-ptmgr will have support for unmanaged pts. But the current design means that anyone who only wants unmanaged pts will still have to carry the code for managed pts.
We might want to adjust `tor-ptmgr` so that managed pluggable transports, and the ipc protocol, are all behind a feature flag.https://gitlab.torproject.org/tpo/core/arti/-/issues/1333Ensure that transport configuration has tests in arti::cfg2024-03-27T15:01:13ZNick MathewsonEnsure that transport configuration has tests in arti::cfgRight now, I think that there's nothing that tests the `[[bridges.transports]]` section in our example configuration.
This is a followup from #755.Right now, I think that there's nothing that tests the `[[bridges.transports]]` section in our example configuration.
This is a followup from #755.Nick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/arti/-/issues/1331Port the ability for directory authorities to compute consensus documents2024-03-12T16:57:09ZGabagaba@torproject.orgPort the ability for directory authorities to compute consensus documentsDirectory authorities need to take the votes and form, encode, compute, and sign the consensus once per hour. This process includes computing a consensus on included relays, descriptors and microdescriptors for each relay, flags and othe...Directory authorities need to take the votes and form, encode, compute, and sign the consensus once per hour. This process includes computing a consensus on included relays, descriptors and microdescriptors for each relay, flags and other properties of each relay, parameters and the weight of these parameters, and recommended versions of Tor.
In this issue, we will port the system for directory authorities to generate votes into Arti.Arti: Directory authorities implementationhttps://gitlab.torproject.org/tpo/core/arti/-/issues/1330Textually encode votes2024-03-12T16:54:05ZGabagaba@torproject.orgTextually encode votesArti: Directory authorities implementationhttps://gitlab.torproject.org/tpo/core/arti/-/issues/1329Compute microdescriptors for current md consensus versions2024-03-12T16:53:53ZGabagaba@torproject.orgCompute microdescriptors for current md consensus versionsArti: Directory authorities implementationhttps://gitlab.torproject.org/tpo/core/arti/-/issues/1328Compute recommended protocol versions2024-03-12T16:51:40ZGabagaba@torproject.orgCompute recommended protocol versionsArti: Directory authorities implementationhttps://gitlab.torproject.org/tpo/core/arti/-/issues/1327Compute known flags2024-03-12T16:53:20ZGabagaba@torproject.orgCompute known flagsArti: Directory authorities implementationhttps://gitlab.torproject.org/tpo/core/arti/-/issues/1326Compute recommended versions2024-03-12T16:53:02ZGabagaba@torproject.orgCompute recommended versionsArti: Directory authorities implementationhttps://gitlab.torproject.org/tpo/core/arti/-/issues/1325Vote on network parameters2024-03-12T16:52:56ZGabagaba@torproject.orgVote on network parametersArti: Directory authorities implementation