Trac issueshttps://gitlab.torproject.org/legacy/trac/-/issues2020-06-13T15:45:18Zhttps://gitlab.torproject.org/legacy/trac/-/issues/31647Should OBSOLETE and ___invisible configuration obtions be available to GETCONF?2020-06-13T15:45:18ZNick MathewsonShould OBSOLETE and ___invisible configuration obtions be available to GETCONF?Right now, you can use GETCONF on the invisible `___UsingTestNetworkDefaults` or the obsolete `DisableIOCP` -- without any complaint from Tor.
Perhaps Tor should complain, or even reject these requests.Right now, you can use GETCONF on the invisible `___UsingTestNetworkDefaults` or the obsolete `DisableIOCP` -- without any complaint from Tor.
Perhaps Tor should complain, or even reject these requests.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/28856Discard strcmp_len()2020-06-13T15:35:43ZNick MathewsonDiscard strcmp_len()We have a strcmp_len() function that we use extensively in parsecommon. But it's just a messed-up version of strncmp. We can get better performance here if we drop it.We have a strcmp_len() function that we use extensively in parsecommon. But it's just a messed-up version of strncmp. We can get better performance here if we drop it.Tor: 0.4.0.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/28837Use openssl's SHA3 implementations when they are faster.2020-06-13T15:36:52ZNick MathewsonUse openssl's SHA3 implementations when they are faster.OpenSSL now has sha3 support. At least some versions have assembly implementations of the keccak core function. That's our signal to update crypto_digest.c to use openssl for sha3.
This matters for startup, since `keccakf()` is about ...OpenSSL now has sha3 support. At least some versions have assembly implementations of the keccak core function. That's our signal to update crypto_digest.c to use openssl for sha3.
This matters for startup, since `keccakf()` is about 10% of our startup time when we have a cached directory.Tor: 0.4.1.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/28481Tor's startup time is getting slower on Android2020-06-13T15:34:16ZTracTor's startup time is getting slower on AndroidWhen upgrading Briar's Tor binaries from 0.2.9.16 to 0.3.4.8, we noticed a difference in Tor's startup time on older Android phones.
Measuring the startup time of several recent Tor versions revealed an interesting pattern. The time tha...When upgrading Briar's Tor binaries from 0.2.9.16 to 0.3.4.8, we noticed a difference in Tor's startup time on older Android phones.
Measuring the startup time of several recent Tor versions revealed an interesting pattern. The time that elapses between starting the Tor process and the creation of the authentication cookie file hasn't changed across versions, but the time between the creation of the cookie file and the response to the AUTHENTICATE command has changed substantially. (Briar sends the AUTHENTICATE command as soon as the cookie file's created.)
I measured five runs of each version on a Motorola Moto G 4G running Android 5.1. Here are the min and max times in seconds for each version:
|= Tor version =|= Min =|= Max =|
|---------------|-------|-------|
|0.2.9.16|3.5|4.3|
|0.2.9.17|3.5|4.2|
|0.3.0.1-alpha|4.8|13.0|
|0.3.1.1-alpha|9.9|16.2|
|0.3.2.1-alpha|15.3|19.9|
|0.3.3.1-alpha|15.8|18.5|
|0.3.4.1-alpha|16.1|18.4|
|0.3.4.8|16.2|20.9|
|0.3.4.9|16.1|23.7|
The min and max have both increased substantially since 0.2.9, and the distribution has widened. This is having a noticeable impact on how long it takes for Briar to connect to contacts when the app's started.
I'll repeat these experiments on Linux x64 to see whether this is Android-specific.
**Trac**:
**Username**: akwizgranTor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/27173Unstable unit tests in control.controller2019-02-05T21:15:26ZdmrUnstable unit tests in control.controller#27053 brought about some changes to stem's controller's event handling.
Unfortunately the tests for that are a bit unstable - sometimes failing.
Instead of letting that ticket drag on, the work is being moved to a new ticket for better...#27053 brought about some changes to stem's controller's event handling.
Unfortunately the tests for that are a bit unstable - sometimes failing.
Instead of letting that ticket drag on, the work is being moved to a new ticket for better clarity.
The history behind that is mostly in #27053, so refer there.
A few parts were discussed over IRC, however, so that context will be filled in below...
From [ticket:27053#comment:14]
> [...] The good news is that I figured out the python3 mock issue you found. Turns out there's a difference between PyPI and Python3's mock modules. Fixed...
>
> https://gitweb.torproject.org/stem.git/commit/?id=e75cf25
I confirmed that this works for me.
From [ticket:27053#comment:15] (the latest comment at time of writing):
> Hi Dave, think I got it but tough to be sure. Does this do the trick for you?
>
> https://gitweb.torproject.org/stem.git/commit/?id=abc6f29
Tests are still failing as of this revision, still giving `AssertionError: Expected 'mock' to be called once. Called 0 times.` for py27.
It seems to be more stable, now failing at ~5-10% rate instead of 25-40%. //(Rate not scientific)//Damian JohnsonDamian Johnsonhttps://gitlab.torproject.org/legacy/trac/-/issues/27053Check controller's event error handling2020-06-13T08:03:45ZDamian JohnsonCheck controller's event error handlingMike had a great question: 'what happens to events if there's an error?'. On reflection the answer is: nothing good.
Our handle_event method parses events without any kind of error handling...
https://gitweb.torproject.org/stem.git/tre...Mike had a great question: 'what happens to events if there's an error?'. On reflection the answer is: nothing good.
Our handle_event method parses events without any kind of error handling...
https://gitweb.torproject.org/stem.git/tree/stem/control.py#n3876
This would in turn break our event handling thread. Instead we should direct malformed events into a 'broken event' queue.Damian JohnsonDamian Johnsonhttps://gitlab.torproject.org/legacy/trac/-/issues/26647defect: Spec for net/listeners/* doesn't covers HTTPTunnelPort directive or E...2020-06-13T15:27:37ZTracdefect: Spec for net/listeners/* doesn't covers HTTPTunnelPort directive or ExtORPortThere are (rather) new type of listener, that's not covered by 'GETINFO net/listeners/*'.
**Trac**:
**Username**: pyhedgehogThere are (rather) new type of listener, that's not covered by 'GETINFO net/listeners/*'.
**Trac**:
**Username**: pyhedgehogTor: 0.3.2.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/26129Stem should invalidate exit_policy cache for DNS_USELESS event2018-08-04T22:25:16ZdmrStem should invalidate exit_policy cache for DNS_USELESS eventAs originally noted in #25423, a number of ControlPort events indicate that tor has changed its exit policy; stem should use these to invalidate the `exit_policy` cache.
All events mentioned in #25423 appear to be implemented except for ...As originally noted in #25423, a number of ControlPort events indicate that tor has changed its exit policy; stem should use these to invalidate the `exit_policy` cache.
All events mentioned in #25423 appear to be implemented except for the [[event](https://gitweb.torproject.org/torspec.git/tree/control-spec.txt?id=436d08b49fb84aa62d7bc96013002a0c27534bbb#n2634|`DNS_USELESS`)].
(So this ticket is for that.)
==== Severity/priority
This event should be very rare, and there are probably a number of other things consumers would want to do if this event occurred, that are out of scope for this ticket. Overall I believe this is best categorized as Low/Minor.Damian JohnsonDamian Johnsonhttps://gitlab.torproject.org/legacy/trac/-/issues/26112Stem should not use human-readable message of GETCONF 552 response2020-06-13T07:43:50ZdmrStem should not use human-readable message of GETCONF 552 responseBoth `tor` and `stem` use a message format for `GETCONF` `552` errors that differs from [[wording of the control spec's `GETCONF` response](https://gitweb.torproject.org/torspec.git/tree/control-spec.txt?id=436d08b49fb84aa62d7bc96013002a...Both `tor` and `stem` use a message format for `GETCONF` `552` errors that differs from [[wording of the control spec's `GETCONF` response](https://gitweb.torproject.org/torspec.git/tree/control-spec.txt?id=436d08b49fb84aa62d7bc96013002a0c27534bbb#n307|the)].
`GETCONF` output via `tor-prompt`:
```
>>> GETCONF blah
552 Unrecognized configuration key "blah"
```
Stem //expects// this format; see [[https://gitweb.torproject.org/stem.git/tree/stem/response/getconf.py?id=aa692e62bfda5be8b87e463c3c899cb13968d32a#n33|`GetConfResponse`]]:
```
unrecognized_keywords = []
for code, _, line in self.content():
if code == '552' and line.startswith('Unrecognized configuration key "') and line.endswith('"'):
unrecognized_keywords.append(line[32:-1])
if unrecognized_keywords:
raise stem.InvalidArguments('552', 'GETCONF request contained unrecognized keywords: %s' % ', '.join(unrecognized_keywords), unrecognized_keywords)
else:
raise stem.ProtocolError('GETCONF response contained a non-OK status code:\n%s' % self)
```
My `tor` version, for reference:
```
>>> GETINFO version
250-version=0.3.2.10 (git-0edaa32732ec8930)
250 OK
```
==== Historical note
In searching (for potential duplicates) to file this ticket, I happened upon the original ticket: #6114
==== Priority, Severity
I'm marking this as low,minor because it appears the code will work decently well (still raise //an// exception) if the message format changes.
==== Expected behavior
In discussion over `#tor-dev`, it was agreed that `stem` (and generally any controller) shouldn't rely on a specific message. Only the numeric code `552` should be relied upon.Damian JohnsonDamian Johnsonhttps://gitlab.torproject.org/legacy/trac/-/issues/26056Stem should optionally timeout when building circuits2018-05-14T01:05:32ZpastlyStem should optionally timeout when building circuitsI recently found out that `CircuitBuildTimeout 10\nLearnCircuitBuildTimeout 0` doesn't always make your circuit build attempts timeout after 10 seconds. In an attemp to avoid repeating myself, I think the commit message[0] has a good sum...I recently found out that `CircuitBuildTimeout 10\nLearnCircuitBuildTimeout 0` doesn't always make your circuit build attempts timeout after 10 seconds. In an attemp to avoid repeating myself, I think the commit message[0] has a good summary of why.
I would find it immensely useful if stem could optionally take a timeout on its `new_circuit` function that would raise an exception when the timeout is hit. For completeness, I imagine `new_circuit`'s sibling functions like `extend_circuit` should probably also take a timeout.
For the time being, I've implemented a work around in sbws. I am providing the commit that makes the change in case it would be helpful for doing something similar in stem. I think it's messy, gross, and probably needlessly complicated. Do with it what you will :) I hope someday soon I can burn it with fire.
[0]: https://github.com/pastly/simple-bw-scanner/commit/dc63ad05bb78bd81d68046688e6f08a717150b9fDamian JohnsonDamian Johnsonhttps://gitlab.torproject.org/legacy/trac/-/issues/26053controller.signal(stem.Signal.NEWNYM) gives error2018-07-29T21:06:14Zcypherpunkscontroller.signal(stem.Signal.NEWNYM) gives errorWhen write this code
controller.signal(stem.Signal.NEWNYM)
It fires this errro below
E1101:Instance of 'Enum' has no 'NEWNYM' memberWhen write this code
controller.signal(stem.Signal.NEWNYM)
It fires this errro below
E1101:Instance of 'Enum' has no 'NEWNYM' memberDamian JohnsonDamian Johnsonhttps://gitlab.torproject.org/legacy/trac/-/issues/25853Behavior for Controller.get_exit_policy() for tor client2020-06-13T10:00:59ZdmrBehavior for Controller.get_exit_policy() for tor clientIn #25423 it was noticed that...
> [...] the client-only configuration ALWAYS returns this:
> {{{
> >>> GETINFO exit-policy/full
> 551 router_get_my_routerinfo returned NULL
> }}}
#25852 tracks implementing defined behavior in tor and t...In #25423 it was noticed that...
> [...] the client-only configuration ALWAYS returns this:
> {{{
> >>> GETINFO exit-policy/full
> 551 router_get_my_routerinfo returned NULL
> }}}
#25852 tracks implementing defined behavior in tor and the control spec to return `551` in this scenario.
This ticket tracks any necessary changes to `stem.control` for `Controller.get_exit_policy()` to return the appropriate ExitPolicy (perhaps `None`) for tor clients.Damian JohnsonDamian Johnsonhttps://gitlab.torproject.org/legacy/trac/-/issues/25821Stem getconf cache doesn't clear for CONF_CHANGED events; probably should set...2018-05-21T16:32:02ZdmrStem getconf cache doesn't clear for CONF_CHANGED events; probably should set valueAs mentioned in #25423, I found a cache-invalidation bug in `stem.control.Controller`.
The tl;dr is that the `getconf` cache currently doesn't get cleared properly by the listener that intends to clear it, but //clearing// probably isn'...As mentioned in #25423, I found a cache-invalidation bug in `stem.control.Controller`.
The tl;dr is that the `getconf` cache currently doesn't get cleared properly by the listener that intends to clear it, but //clearing// probably isn't what we actually want to do.dmrdmrhttps://gitlab.torproject.org/legacy/trac/-/issues/25511Expose TZ info on control port for better debugging of time errors2020-06-13T17:44:06ZNick MathewsonExpose TZ info on control port for better debugging of time errorsWhen we tell people that their clocks are wrong, it's frequently because they've set up their systems with the wrong time zone. It would be helpful to tell torbrowser (or some other controller) about this information, so that it can giv...When we tell people that their clocks are wrong, it's frequently because they've set up their systems with the wrong time zone. It would be helpful to tell torbrowser (or some other controller) about this information, so that it can give more useful error messages on time-related bootstrapping failures.
~~One complication here is (IIUC) TB runs, and runs tor, with its TZ set to UTC.~~
Further investigation suggests that TB might not set `TZ` for the first time it starts tor. Opened #25823 for the Tor Launcher behavior inconsistency.Tor: 0.3.4.x-finalNeel Chauhanneel@neelc.orgNeel Chauhanneel@neelc.orghttps://gitlab.torproject.org/legacy/trac/-/issues/22627asyncio controller2019-12-20T23:22:39ZDamian Johnsonasyncio controllerDamn folks seem excited about asyncio. Seems to be a builtin intended to replace twisted starting with Python 3.4...
https://docs.python.org/3/library/asyncio.html
We should consider adding an AsyncController class that uses it, so we ...Damn folks seem excited about asyncio. Seems to be a builtin intended to replace twisted starting with Python 3.4...
https://docs.python.org/3/library/asyncio.html
We should consider adding an AsyncController class that uses it, so we can integrate with folks who want a controller using that pattern.Damian JohnsonDamian Johnsonhttps://gitlab.torproject.org/legacy/trac/-/issues/21329onions/detached and onions/current should return empty lists instead of errors2017-05-15T19:44:31Zmeejahonions/detached and onions/current should return empty lists instead of errorsIf you do a GETINFO query on `onions/detached` or `onions/current` and there aren't any, a 551 error is returned. Most other things just return an empty list/value in this case and I think these should too.
One consequence of this is th...If you do a GETINFO query on `onions/detached` or `onions/current` and there aren't any, a 551 error is returned. Most other things just return an empty list/value in this case and I think these should too.
One consequence of this is that if you ask for `onions/current` with some other keys and there are no current onions, the whole query will fail.
So `GETINFO onions/current` should return `onions/current=` instead of an error.
For example you can try `carml cmd getinfo orconn-status onions/current` to see this behavior.Tor: 0.3.1.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/19925SR: Control event to get the shared random values2020-06-13T15:00:18ZDavid Gouletdgoulet@torproject.orgSR: Control event to get the shared random valuesOnce all dirauth move to 029, the consensus will have a shared random value regurlarly. Having a way to get that value through the control port would be useful for many purposes including people wanting to use that value outside of tor.
...Once all dirauth move to 029, the consensus will have a shared random value regurlarly. Having a way to get that value through the control port would be useful for many purposes including people wanting to use that value outside of tor.
I recommend we implement that only in 030 so we'll have a full stable version to debug and make sure it actually works properly before we expose it to a wider audience.
This ticket is the main one for both the spec and implementation.Tor: 0.3.0.x-finalDavid Gouletdgoulet@torproject.orgDavid Gouletdgoulet@torproject.orghttps://gitlab.torproject.org/legacy/trac/-/issues/19609Wrong version restriction for ADD_ONION_BASIC_AUTH2020-06-13T14:59:21ZTracWrong version restriction for ADD_ONION_BASIC_AUTHADD_ONION_BASIC_AUTH appears to be restricted to Tor 0.2.9.1-alpha and above in version.py (https://gitweb.torproject.org/stem.git/tree/stem/version.py#n368) but correct version is 0.2.9.0-alpha (latest version in Tor Git).
It's probab...ADD_ONION_BASIC_AUTH appears to be restricted to Tor 0.2.9.1-alpha and above in version.py (https://gitweb.torproject.org/stem.git/tree/stem/version.py#n368) but correct version is 0.2.9.0-alpha (latest version in Tor Git).
It's probably a typo in version.py, needs to be 0.2.9.0 instead of 0.2.9.1.
**Trac**:
**Username**: drazvanDamian JohnsonDamian Johnsonhttps://gitlab.torproject.org/legacy/trac/-/issues/18920Make consensus GETINFO return 551 when using microdescs2020-06-13T14:56:49ZteorMake consensus GETINFO return 551 when using microdescsThe tor control-spec says that 551 should be returned for "dir/status" queries where microdescs are in use, and so full descriptors are unavailable.
https://gitweb.torproject.org/torspec.git/tree/control-spec.txt#n737
This doesn't happe...The tor control-spec says that 551 should be returned for "dir/status" queries where microdescs are in use, and so full descriptors are unavailable.
https://gitweb.torproject.org/torspec.git/tree/control-spec.txt#n737
This doesn't happen for dir/status-vote/current/consensus when tor only has a microdesc consensus, instead, tor returns "552 Unrecognized key "dir/status-vote/current/consensus".
https://gitweb.torproject.org/tor.git/tree/src/or/control.c#n2004
The 552 is misleading and not to spec, instead, we should return 551 with a helpful error message.
Credit to s0rlxmh0 for reporting this issue.Tor: 0.2.8.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/16348Suppress exception chaining when PEP 3134 is merged2019-12-20T23:25:37ZtoralfSuppress exception chaining when PEP 3134 is mergedwith tor-0.2.6.9 and stem-1.4.1 I run (rarely) into this :
```
cat ioerror.stderr.old
Exception in thread Event Notifier:
Traceback (most recent call last):
File "/usr/lib64/python3.3/site-packages/stem/control.py", line 1758, in get_n...with tor-0.2.6.9 and stem-1.4.1 I run (rarely) into this :
```
cat ioerror.stderr.old
Exception in thread Event Notifier:
Traceback (most recent call last):
File "/usr/lib64/python3.3/site-packages/stem/control.py", line 1758, in get_network_status
desc_content = self.get_info(query, get_bytes = True)
File "/usr/lib64/python3.3/site-packages/stem/control.py", line 414, in wrapped
raise exc
File "/usr/lib64/python3.3/site-packages/stem/control.py", line 409, in wrapped
return func(self, *args, **kwargs)
File "/usr/lib64/python3.3/site-packages/stem/control.py", line 1113, in get_info
raise exc
File "/usr/lib64/python3.3/site-packages/stem/control.py", line 1066, in get_info
stem.response.convert('GETINFO', response)
File "/usr/lib64/python3.3/site-packages/stem/response/__init__.py", line 135, in convert
message._parse_message(**kwargs)
File "/usr/lib64/python3.3/site-packages/stem/response/getinfo.py", line 38, in _parse_message
raise stem.InvalidArguments('552', 'GETINFO request contained unrecognized keywords: %s\n' % ', '.join(unrecognized_keywords), unrecognized_keywords)
stem.InvalidArguments: GETINFO request contained unrecognized keywords: ns/id/2BCDF9F0BCEFC2A44F7850F92362BA85AA226E1F
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib64/python3.3/threading.py", line 901, in _bootstrap_inner
self.run()
File "/usr/lib64/python3.3/threading.py", line 858, in run
self._target(*self._args, **self._kwargs)
File "/usr/lib64/python3.3/site-packages/stem/control.py", line 882, in _event_loop
self._handle_event(event_message)
File "/usr/lib64/python3.3/site-packages/stem/control.py", line 3480, in _handle_event
listener(event_message)
File "./err.py", line 47, in orconn_event
relay = controller.get_network_status(fingerprint)
File "/usr/lib64/python3.3/site-packages/stem/control.py", line 414, in wrapped
raise exc
File "/usr/lib64/python3.3/site-packages/stem/control.py", line 409, in wrapped
return func(self, *args, **kwargs)
File "/usr/lib64/python3.3/site-packages/stem/control.py", line 1761, in get_network_status
raise stem.DescriptorUnavailable("Tor was unable to provide the descriptor for '%s'" % relay)
stem.DescriptorUnavailable: Tor was unable to provide the descriptor for '2BCDF9F0BCEFC2A44F7850F92362BA85AA226E1F'
```
while running this script :
```
$ cat err.py
#!/usr/bin/python3 -u
# Toralf Foerster
# Hamburg
# Germany
# collect data wrt to https://trac.torproject.org/projects/tor/ticket/13603
#
import time
import functools
from stem import ORStatus, ORClosureReason
from stem.control import EventType, Controller
def main():
class Cnt(object):
def __init__(self, done=0, closed=0, ioerror=0):
self.done = done
self.closed = closed
self.ioerror = ioerror
c = Cnt()
with Controller.from_port(port=9051) as controller:
controller.authenticate()
orconn_listener = functools.partial(orconn_event, controller, c)
controller.add_event_listener(orconn_listener, EventType.ORCONN)
while True:
time.sleep(1)
def orconn_event(controller, c, event):
if event.status == ORStatus.CLOSED:
c.closed += 1
if event.reason == ORClosureReason.DONE:
c.done += 1
if event.reason == ORClosureReason.IOERROR:
c.ioerror += 1
fingerprint = event.endpoint_fingerprint
print (" %i %i %i %i %s %40s" % (c.closed, c.done, c.ioerror, event.arrived_at, time.ctime(event.arrived_at), fingerprint), end='')
relay = controller.get_network_status(fingerprint)
if relay:
print (" %15s %5i %s %s" % (relay.address, relay.or_port, controller.get_info("ip-to-country/%s" % relay.address, 'unknown'), relay.nickname), end='')
print ('', flush=True)
if __name__ == '__main__':
main()
```Damian JohnsonDamian Johnson