The Tor Project issueshttps://gitlab.torproject.org/groups/tpo/-/issues2022-03-01T17:27:14Zhttps://gitlab.torproject.org/tpo/core/arti/-/issues/311Extreme CPU usage2022-03-01T17:27:14ZmohixExtreme CPU usage<!--
* Use this issue template for reporting a new bug.
-->
### Summary
Im having multiple WAN ports which every 5 minutes the router switch between them in a round robin algorithmic way. This happen inside the router.
After WAN switch ...<!--
* Use this issue template for reporting a new bug.
-->
### Summary
Im having multiple WAN ports which every 5 minutes the router switch between them in a round robin algorithmic way. This happen inside the router.
After WAN switch happen arti lose the connection with it peers which its completely normal but after trying to make new peers its CPU usage goes higher. After 7-8 switch arti use 100% of CPU. I guess there its some lock that prevents the lost connection to finish and destruct it, rather than its still trying to make the connection on the lost port.
### Steps to reproduce:
You dont need high configure system to test it. Just have two different internet WAN its enough.
1. Remove the ethernet WAN port from PC and put another ethernet WAN.
2. Wait until atri make new connections.
3. Repeat steep [1,2] for 10 times.
### What is the current bug behavior?
Using 100% of CPU usage
### What is the expected behavior?
CPU usage should be less than 1%
### Environment
- Version: Arti 0.0.4
- Operating system: Windows 11
- Install method: from git
- etc...
### Relevant logs and/or screenshots:
<details><summary>Log</summary>
←[2m2022-02-01T09:24:42.805661Z←[0m ←[33m WARN←[0m ←[2mtor_proto::circuit::reactor←[0m←[2m:←[0m Circ 2.5: having to enqueue cell due to backpressure: ChanCell { circid: CircId(2363215174), msg: Relay(Relay) }
←[2m2022-02-01T09:24:42.830781Z←[0m ←[33m WARN←[0m ←[2mtor_proto::circuit::reactor←[0m←[2m:←[0m Circ 2.5: having to enqueue cell due to backpressure: ChanCell { circid: CircId(2363215174), msg: Relay(Relay) }
←[2m2022-02-01T09:24:42.840138Z←[0m ←[33m WARN←[0m ←[2mtor_proto::circuit::reactor←[0m←[2m:←[0m Circ 2.5: having to enqueue cell due to backpressure: ChanCell { circid: CircId(2363215174), msg: Relay(Relay) }
←[2m2022-02-01T09:24:42.884217Z←[0m ←[33m WARN←[0m ←[2mtor_proto::circuit::reactor←[0m←[2m:←[0m Circ 2.5: having to enqueue cell due to backpressure: ChanCell { circid: CircId(2363215174), msg: Relay(Relay) }
←[2m2022-02-01T09:24:42.922146Z←[0m ←[33m WARN←[0m ←[2mtor_proto::circuit::reactor←[0m←[2m:←[0m Circ 2.5: having to enqueue cell due to backpressure: ChanCell { circid: CircId(2363215174), msg: Relay(Relay) }
←[2m2022-02-01T09:25:11.716475Z←[0m ←[33m WARN←[0m ←[2marti::proxy←[0m←[2m:←[0m connection exited with error: directory timed out
←[2m2022-02-01T09:25:11.745268Z←[0m ←[33m WARN←[0m ←[2marti::proxy←[0m←[2m:←[0m connection exited with error: directory timed out
←[2m2022-02-01T09:25:12.204730Z←[0m ←[33m WARN←[0m ←[2marti::proxy←[0m←[2m:←[0m connection exited with error: directory timed out
←[2m2022-02-01T09:27:47.633435Z←[0m ←[33m WARN←[0m ←[2marti::proxy←[0m←[2m:←[0m connection exited with error: Error while reading SOCKS handshake
←[2m2022-02-01T09:27:47.633433Z←[0m ←[33m WARN←[0m ←[2marti::proxy←[0m←[2m:←[0m connection exited with error: Error while reading SOCKS handshake
←[2m2022-02-01T09:27:47.643401Z←[0m ←[33m WARN←[0m ←[2marti::proxy←[0m←[2m:←[0m connection exited with error: Error while reading SOCKS handshake
</details>
### Possible fixes:
Restarting arti makes it normalArti 1.0.0: Ready for production usehttps://gitlab.torproject.org/tpo/core/arti/-/issues/306Audit usage of SystemTime::now() outside of tests, rtcompat.2022-06-15T13:56:17ZNick MathewsonAudit usage of SystemTime::now() outside of tests, rtcompat.We should see where we are calling `SystemTime::now()` directly, and in most cases we should instead call `SleepProvider::wallclock()` or take `now` as an argument.
See !268 for an example of a bug caused by not mocking the view of the ...We should see where we are calling `SystemTime::now()` directly, and in most cases we should instead call `SleepProvider::wallclock()` or take `now` as an argument.
See !268 for an example of a bug caused by not mocking the view of the current time.Arti 1.0.0: Ready for production usehttps://gitlab.torproject.org/tpo/core/arti/-/issues/133How-to documentation for getting existing programs to use Arti2022-03-01T17:27:13ZNick MathewsonHow-to documentation for getting existing programs to use ArtiWe should write some simple example programs, and some how-to documentation, showing how to use our tor-client API to .
Probably these examples should include something trivial like netcat, and something more applied, like using some po...We should write some simple example programs, and some how-to documentation, showing how to use our tor-client API to .
Probably these examples should include something trivial like netcat, and something more applied, like using some popular libraries over arti connections.Arti 1.0.0: Ready for production usehttps://gitlab.torproject.org/tpo/core/arti/-/issues/314Should the [system] configuration belong to `arti`, not `arti-client`?2022-03-23T11:43:19ZNick MathewsonShould the [system] configuration belong to `arti`, not `arti-client`?Right now, the `[system]` configuration heading is used for the `max_file_descriptors` value, which we set in `arti`. But the section itself is part of `TorClientConfig`.
This violates the encapsulation we have in mind: if there is fun...Right now, the `[system]` configuration heading is used for the `max_file_descriptors` value, which we set in `arti`. But the section itself is part of `TorClientConfig`.
This violates the encapsulation we have in mind: if there is functionality that gets configured in `TorClientConfig`, it should be implemented in `arti-client` or below so that all Arti users get it.
Personally I think it makes more sense to have `max_file_descriptors` implemented in `arti`: it isn't very usual for libraries to mess with their applications' resource limits. If that's the case, it doesn't belong in `TorClientConfig`.
We might want to defer this change until after #285 is done, since that ticket will substantially refactor our configuration.Arti 1.0.0: Ready for production usehttps://gitlab.torproject.org/tpo/core/arti/-/issues/265Do half-open streams behave correctly for RESOLVE/RESOLVED and similar commands?2022-03-01T17:27:04ZNick MathewsonDo half-open streams behave correctly for RESOLVE/RESOLVED and similar commands?I'm not 100% sure that we call terminate() right—or that we should call it at all!—for streams that we have created by sending a cell other than BEGIN or BEGINDIR. On those streams, if we discard them, we shouldn't allow "maybe a connec...I'm not 100% sure that we call terminate() right—or that we should call it at all!—for streams that we have created by sending a cell other than BEGIN or BEGINDIR. On those streams, if we discard them, we shouldn't allow "maybe a connected and a big pile of data cells": we should instead allow only the cells that would be allowed if the stream were still open.Arti 1.0.0: Ready for production usehttps://gitlab.torproject.org/tpo/core/arti/-/issues/264Expire half-open streams, possibly using a timer?2022-03-01T17:27:03ZNick MathewsonExpire half-open streams, possibly using a timer?Right now, when we send an END on a stream, there might be potential for that stream's `StreamEnt::HalfSent` to stick around indefinitely (or at least, for the lifetime of the circuit) if the other side doesn't also send an END.
Perhaps...Right now, when we send an END on a stream, there might be potential for that stream's `StreamEnt::HalfSent` to stick around indefinitely (or at least, for the lifetime of the circuit) if the other side doesn't also send an END.
Perhaps we should expire these on a timer or something; we should check what Tor does.Arti 1.0.0: Ready for production usehttps://gitlab.torproject.org/tpo/core/arti/-/issues/305Upgrade to Rustls 0.202022-11-27T19:28:33ZNick MathewsonUpgrade to Rustls 0.20Right now we're on Rustls 0.19; it would probably be clever to move to Rustls 0.20 if that's viable.Right now we're on Rustls 0.19; it would probably be clever to move to Rustls 0.20 if that's viable.Arti 1.0.0: Ready for production usehttps://gitlab.torproject.org/tpo/core/arti/-/issues/91Parallize bootstrapping download attempts2022-03-01T17:26:54ZNick MathewsonParallize bootstrapping download attemptsIn C tor, we found that it's a good idea for clients to launch multiple initial download attempts in parallel, until they find a circuit that can really give them a consensus. Arti should probably do the same, so that users don't run in...In C tor, we found that it's a good idea for clients to launch multiple initial download attempts in parallel, until they find a circuit that can really give them a consensus. Arti should probably do the same, so that users don't run into the same bootstrapping problems.Arti 1.0.0: Ready for production usedagondagonhttps://gitlab.torproject.org/tpo/core/arti/-/issues/63Implement circuit padding machines2024-03-05T14:36:52ZNick MathewsonImplement circuit padding machinesC tor has functionality for sending and receiving padding cells on circuits according to configurable padding machines. Eventually we may want to implement that in Arti.
* [ ] Implement (or select) numerical estimators for relevant pr...C tor has functionality for sending and receiving padding cells on circuits according to configurable padding machines. Eventually we may want to implement that in Arti.
* [ ] Implement (or select) numerical estimators for relevant probability distributions (prob_distr.c) (50h)
* [ ] Provide relevant events from arti to padding system (circ started, circ opened, stream started, stream ended, cell sent/recv, out of relay early cells) (40h)
* [ ] Implement state machine descriptions using events + distributions + transitions (60h)
* [ ] Implement handling state machine transitions using a state machine as input (60h)
* [ ] Implement padding machine selection negotiation (40h)
* [ ] Implement padding overhead rate limiting using consensus parameters (20h)https://gitlab.torproject.org/tpo/core/arti/-/issues/135See if we can get on OSS-Fuzz2022-02-24T19:40:33ZNick MathewsonSee if we can get on OSS-FuzzTor is currently supported by Google's OSS-Fuzz project; we should see if we can get Arti supported as well.Tor is currently supported by Google's OSS-Fuzz project; we should see if we can get Arti supported as well.https://gitlab.torproject.org/tpo/core/arti/-/issues/64Implement relevant network parameters2022-02-22T16:00:28ZNick MathewsonImplement relevant network parametersTo avoid some kinds of distinguishability and incorrectness, we should make sure we obey all the relevant network parameters.
No estimate here yet, since we need to inventory them first.To avoid some kinds of distinguishability and incorrectness, we should make sure we obey all the relevant network parameters.
No estimate here yet, since we need to inventory them first.Arti 1.0.0: Ready for production usehttps://gitlab.torproject.org/tpo/core/arti/-/issues/65Implement path-bias detection (if still needed?)2023-11-13T16:30:51ZNick MathewsonImplement path-bias detection (if still needed?)If we haven't got a better relay encryption mechanism by the time we want to call Arti secure-enough, we'll need to implement the pathbias mechanism that Tor does in order to prevent some kinds of destructive tagging attacks.If we haven't got a better relay encryption mechanism by the time we want to call Arti secure-enough, we'll need to implement the pathbias mechanism that Tor does in order to prevent some kinds of destructive tagging attacks.Arti: Feature parity with the C implementationhttps://gitlab.torproject.org/tpo/core/arti/-/issues/68Fairness and queueing for circuits, streams, and channels2022-02-22T16:14:43ZNick MathewsonFairness and queueing for circuits, streams, and channelsToday's arti uses a greedy strategy for reading cells and data. Between now and when we're done, we should do something closer to Tor's current approach, which uses a mix of round-robining, cell queueing, ewma circuit scheduling, KIST, ...Today's arti uses a greedy strategy for reading cells and data. Between now and when we're done, we should do something closer to Tor's current approach, which uses a mix of round-robining, cell queueing, ewma circuit scheduling, KIST, etc, etc.
We should probably break out some of the above features for later deployment, since they're more important for relays and onion services than for clients.Arti: Feature parity with the C implementationhttps://gitlab.torproject.org/tpo/core/arti/-/issues/71Support for "Dormant" mode for mobile/embedded clients2023-11-15T19:02:30ZNick MathewsonSupport for "Dormant" mode for mobile/embedded clientsIf walking onions isn't implemented by the time arti is in general use, we should try to have a "dormant mode" available to avoid needless wakeups, directory downloads, and circuit construction.If walking onions isn't implemented by the time arti is in general use, we should try to have a "dormant mode" available to avoid needless wakeups, directory downloads, and circuit construction.Arti: Feature parity with the C implementationhttps://gitlab.torproject.org/tpo/core/arti/-/issues/72Run as more kinds of proxy2022-12-19T18:41:25ZNick MathewsonRun as more kinds of proxyTor can run as these kinds of proxy:
* transparent
* http connect
* dns
We should do some of these in Arti, depending on demand.Tor can run as these kinds of proxy:
* transparent
* http connect
* dns
We should do some of these in Arti, depending on demand.Arti: Feature parity with the C implementationhttps://gitlab.torproject.org/tpo/core/arti/-/issues/77Improve buffering and maybe performance on DataStream2022-02-24T14:51:30ZNick MathewsonImprove buffering and maybe performance on DataStreamThe current DataStream implementation isn't optimized at all. That's well and good for now, but eventually we should refactor it to have better performance, possibly buffer more data, but buffer no more than needed, etc.The current DataStream implementation isn't optimized at all. That's well and good for now, but eventually we should refactor it to have better performance, possibly buffer more data, but buffer no more than needed, etc.Arti 1.0.0: Ready for production usehttps://gitlab.torproject.org/tpo/core/arti/-/issues/70Backward compatibility with some (or all?) of controller protocol?2023-11-13T16:30:06ZNick MathewsonBackward compatibility with some (or all?) of controller protocol?**If** we do this, the subtasks will be more or less as follows:
* Support for parsing and encoding the meta-format of the control protocol (24h)
* Support for one or more authentication mechanisms (24h)
* Support for opening and l...**If** we do this, the subtasks will be more or less as follows:
* Support for parsing and encoding the meta-format of the control protocol (24h)
* Support for one or more authentication mechanisms (24h)
* Support for opening and listening on a control port (8h)
* Unix socket support (8h)
* Support for maintaining and using a controller cookie. (8h)
Then for each command type that we indend to support, we need to:
* Make sure backend support exists
* Implement the command parsing and response
For each event we intend to support, we need to:
* Make sure it's exposed via our asynchronous events mechanism #357
* Tie it into the controlport logic.https://gitlab.torproject.org/tpo/core/arti/-/issues/97Security audit for Arti2022-02-22T15:57:21ZNick MathewsonSecurity audit for ArtiIf we can find the money, we should pay somebody do to a security audit on Arti before we recommend it for production use.If we can find the money, we should pay somebody do to a security audit on Arti before we recommend it for production use.Arti: Feature parity with the C implementationhttps://gitlab.torproject.org/tpo/core/arti/-/issues/99Path-limiting options as supported in Tor2023-11-15T19:02:53ZNick MathewsonPath-limiting options as supported in TorSomeday we may want Tor's full suite of Exclude*Nodes options, and similar. Not soon, though, I hope.Someday we may want Tor's full suite of Exclude*Nodes options, and similar. Not soon, though, I hope.Arti: Feature parity with the C implementationhttps://gitlab.torproject.org/tpo/core/arti/-/issues/101Supported C FFI for Arti2023-11-15T18:57:57ZNick MathewsonSupported C FFI for ArtiWe should declare a C API for Arti that others can use. It might not need to start with the full functionality of the Rust API.
To avoid sending others down a big trip into FuturesLand, design will be needed. Maybe the right thing to ...We should declare a C API for Arti that others can use. It might not need to start with the full functionality of the Rust API.
To avoid sending others down a big trip into FuturesLand, design will be needed. Maybe the right thing to do is provide a set of blocking APIs, and/or expose poll-style APIs?Arti: RPC Support