Trac issueshttps://gitlab.torproject.org/legacy/trac/-/issues2020-06-16T01:12:09Zhttps://gitlab.torproject.org/legacy/trac/-/issues/30882Stem python 3.8 RuntimeError: dictionary keys changed during iteration2020-06-16T01:12:09ZteorStem python 3.8 RuntimeError: dictionary keys changed during iterationpython 3.8 doesn't seem to like dictionary keys changing during iteration. This is on python nightly, so it might be a new python feature (or even a python bug).
It is reproducible: I've seen it in at least two runs so far.
```
Jun 1...python 3.8 doesn't seem to like dictionary keys changing during iteration. This is on python nightly, so it might be a new python feature (or even a python bug).
It is reproducible: I've seen it in at least two runs so far.
```
Jun 13 15:02:32.725 [notice] Bootstrapped 100% (done): Done
done (17 seconds)
Shutting down tor... done
Traceback (most recent call last):
File "./run_tests.py", line 479, in <module>
main()
File "./run_tests.py", line 306, in main
integ_runner.start(target, args.attribute_targets, args.tor_path)
File "/home/travis/build/torproject/stem/test/runner.py", line 262, in start
self._owner_controller = self.get_tor_controller(True)
File "/home/travis/build/torproject/stem/test/runner.py", line 482, in get_tor_controller
controller.authenticate(password = CONTROL_PASSWORD, chroot_path = self.get_chroot())
File "/home/travis/build/torproject/stem/stem/control.py", line 1110, in authenticate
stem.connection.authenticate(self, *args, **kwargs)
File "/home/travis/build/torproject/stem/stem/connection.py", line 596, in authenticate
controller._post_authentication()
File "/home/travis/build/torproject/stem/stem/control.py", line 3993, in _post_authentication
owning_pid = self.get_conf('__OwningControllerProcess', None)
File "/home/travis/build/torproject/stem/stem/control.py", line 2261, in get_conf
entries = self.get_conf_map(param, default, multiple)
File "/home/travis/build/torproject/stem/stem/control.py", line 2364, in get_conf_map
for key in reply:
RuntimeError: dictionary keys changed during iteration
The command "./run_tests.py --integ $TARGET" exited with 1.
```
https://travis-ci.org/torproject/stem/jobs/545118990Damian JohnsonDamian Johnsonhttps://gitlab.torproject.org/legacy/trac/-/issues/21177Add support for parsing bridge-distribution-request lines in Stem2020-06-13T18:29:00ZIsis LovecruftAdd support for parsing bridge-distribution-request lines in StemFrom #18329, we added a new bridge-server-descriptor line: `bridge-distribution-request`, which, if set to "none" should tell BridgeDB not to distribute a bridge. This allows, for example, BridgeDB to avoid distributing the default Tor ...From #18329, we added a new bridge-server-descriptor line: `bridge-distribution-request`, which, if set to "none" should tell BridgeDB not to distribute a bridge. This allows, for example, BridgeDB to avoid distributing the default Tor Browser bridges, and allows other bridges to not be distributed, but to still submit their usage statistics for Metrics.Damian JohnsonDamian Johnsonhttps://gitlab.torproject.org/legacy/trac/-/issues/15004Hidden service descriptor parsing2020-06-13T18:13:25ZDamian JohnsonHidden service descriptor parsingMonths ago asn asked us for this but I said 'not until tor provides a way of actually... well, getting it'. With #14847 this is now becoming a thing so it's time stem provided a parser.
David, would you mind adding an example to this ti...Months ago asn asked us for this but I said 'not until tor provides a way of actually... well, getting it'. With #14847 this is now becoming a thing so it's time stem provided a parser.
David, would you mind adding an example to this ticket for what a hs descriptor in the wild looks like? I'll use it for stem's unit test.Damian JohnsonDamian Johnsonhttps://gitlab.torproject.org/legacy/trac/-/issues/28495Please provide a parser for detached signatures2020-06-13T17:58:15ZirlPlease provide a parser for detached signaturesThe format for these is described in dir-spec §3.10.The format for these is described in dir-spec §3.10.Damian JohnsonDamian Johnsonhttps://gitlab.torproject.org/legacy/trac/-/issues/8049Stem's DescriptorReader misses 10% of descriptors in tarballs2020-06-13T17:50:08ZKarsten LoesingStem's DescriptorReader misses 10% of descriptors in tarballsWhen I parse server descriptors in a metrics tarball and in extracted form using DescriptorReader, I get different numbers of descriptors:
```
from stem.descriptor.reader import DescriptorReader
descriptors = 0
with DescriptorReader('s...When I parse server descriptors in a metrics tarball and in extracted form using DescriptorReader, I get different numbers of descriptors:
```
from stem.descriptor.reader import DescriptorReader
descriptors = 0
with DescriptorReader('server-descriptors-2012-12.tar') as reader:
for descriptor in reader:
descriptors += 1
print "%d descriptors in tarball." % (descriptors, )
descriptors = 0
with DescriptorReader('server-descriptors-2012-12/') as reader:
for descriptor in reader:
descriptors += 1
print "%d descriptors in extracted directory." % (descriptors, )
```
250048 descriptors in tarball.
279042 descriptors in extracted directory.
What happens to the rest?
This bug means that most of #7828 must be re-run. :(Damian JohnsonDamian Johnsonhttps://gitlab.torproject.org/legacy/trac/-/issues/27564Implement Collapsible FAQ items2020-06-13T17:26:58ZTracImplement Collapsible FAQ itemsFollowing Nyx's approach, collapsible FAQ items are implemented as a static html file that gets loaded by Sphinx without being built.
This ticket closes issue: #19679
https://github.com/elias-pap/stem/commit/25a599ba847c29c0f8d6d1570e2...Following Nyx's approach, collapsible FAQ items are implemented as a static html file that gets loaded by Sphinx without being built.
This ticket closes issue: #19679
https://github.com/elias-pap/stem/commit/25a599ba847c29c0f8d6d1570e2a56b437d6f08b
**Trac**:
**Username**: eliaspapDamian JohnsonDamian Johnsonhttps://gitlab.torproject.org/legacy/trac/-/issues/24473People page edit - remove Linda2020-06-13T17:25:47ZErin WyattPeople page edit - remove Linda```
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512
Please remove Linda’s listing on the people page. She is no longer employed by TPI and does not want to continue working with the CC for the time being.
https://www.torproject.org/a...```
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512
Please remove Linda’s listing on the people page. She is no longer employed by TPI and does not want to continue working with the CC for the time being.
https://www.torproject.org/about/corepeople.html.en#linda
-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - https://gpgtools.org
iQIzBAEBCgAdFiEENecqn2ZVRfkstmYkugyUAPgPkc4FAlofMHgACgkQugyUAPgP
kc7f3RAAgpYMc+aSJgYERQjM+dt7fbpkBGF2CM6qv2jLtBHx4gLb0k5bj9EPo2ZD
65tPQ2zFqAwKwGL/NcA3qAGF8pdxzV0yzrKxHqijBL02jIebanOlkK122d2T2h4w
2vsUGB6e3YQN08k5jqA91J9j2w815lxITHoH7m6+Zz2hbFDJAsUa3gG5x1nr785n
Jvkf0D0kExm4Fe0TfmQ3B1x07SU6JegFWkTW+jRQHoGkmet48iohyN2pBGE3qQKX
LU3x1GXzd1ZAGLX1xtDumcT+0Kow1D5A80A9Fh9yGXiwGqSTC8gsXFiK4EE77rGi
yw23VMPxXF2vBDgxfqx/PWLhagPXz6WD4fbLqMbUy5YAiMiqyiXdMLPU/DzPYyYa
+Ny8dgzuey2SM5I82YQRqcu9l7ZsCoV0C/NU7H6kMtgh+fnWoGOUscub8pNJRtk9
llHJMMfEIs8cBTZ3TdGje6pE6155+TwXWS9NKP39JYxgFffOesxLduWj4qOrF3w9
WuhHG+O6McIB1hVJdB8jOdOenWXJV7SKzqBrdvCxEMHJdZZ5/tNz8bSziOIjiz7q
GhbHWo/cSXNbVbzOPuCtOx75+F5J53leOa1IF7LNeU+O1spdgX6P+WP2vucyhXSy
l2jTy5usVIa1XeJSnefAnP14bWPooRTnlk8Gg6IjSR6j4Igs6Zs=
=Po0t
-----END PGP SIGNATURE-----
```Damian JohnsonDamian Johnsonhttps://gitlab.torproject.org/legacy/trac/-/issues/21593drop tor2web from the website lists2020-06-13T17:24:27ZRoger Dingledinedrop tor2web from the website listsWe have a bunch of people who are concerned with tor2web, and think it needs to disappear. Maybe it does, maybe it doesn't, but I think it's fair to stop listing it as a Tor project on our website.
I think that's the official projects l...We have a bunch of people who are concerned with tor2web, and think it needs to disappear. Maybe it does, maybe it doesn't, but I think it's fair to stop listing it as a Tor project on our website.
I think that's the official projects list, and also the table on the volunteer page.Damian JohnsonDamian Johnsonhttps://gitlab.torproject.org/legacy/trac/-/issues/19360FlashProxy is archived in the Trac, but we still seek volunteers2020-06-13T17:23:55ZirlFlashProxy is archived in the Trac, but we still seek volunteersWhile going through /getinvolved/volunteer.html.en for broken links, I noticed that FlashProxy is listed as archived in trac. Do we still want to list that in the projects we'd like volunteers for?While going through /getinvolved/volunteer.html.en for broken links, I noticed that FlashProxy is listed as archived in trac. Do we still want to list that in the projects we'd like volunteers for?Damian JohnsonDamian Johnsonhttps://gitlab.torproject.org/legacy/trac/-/issues/18544Please add Günes Acar and boklm to the Panopticlick mentor list2020-06-13T17:23:41ZGeorg KoppenPlease add Günes Acar and boklm to the Panopticlick mentor listGünes Acar and Nicolas agreed to be (backup) mentors for the Panopticlick GSoC project. Please add them on the webpage.Günes Acar and Nicolas agreed to be (backup) mentors for the Panopticlick GSoC project. Please add them on the webpage.Damian JohnsonDamian Johnsonhttps://gitlab.torproject.org/legacy/trac/-/issues/16575Add chutney and exitmap as projects on the volunteer page2020-06-13T17:23:06ZRoger DingledineAdd chutney and exitmap as projects on the volunteer pageWe were looking through
https://www.torproject.org/getinvolved/volunteer#projects
tonight and noticed that chutney and exitmap are interesting reasonably active projects that aren't on the list.
(Trunnel might also be in this category o...We were looking through
https://www.torproject.org/getinvolved/volunteer#projects
tonight and noticed that chutney and exitmap are interesting reasonably active projects that aren't on the list.
(Trunnel might also be in this category one day. I don't know if it is yet. Probably not yet.)Damian JohnsonDamian Johnsonhttps://gitlab.torproject.org/legacy/trac/-/issues/3674website frontpage implies there are exactly 4 tor projects2020-06-13T17:19:22ZRoger Dingledinewebsite frontpage implies there are exactly 4 tor projectsWe should (for example) have some page that lists more than just the four, and link to it from the frontpage.We should (for example) have some page that lists more than just the four, and link to it from the frontpage.Damian JohnsonDamian Johnsonhttps://gitlab.torproject.org/legacy/trac/-/issues/22301make test-stem doesn't work2020-06-13T16:53:04ZTaylor Yumake test-stem doesn't workRunning `make test-stem` doesn't seem to work on master (tor master with stem master). This probably prevents people from running `make test-full`, which `CodingStandards.md` recommends.
There are probably multiple bugs involved. (I c...Running `make test-stem` doesn't seem to work on master (tor master with stem master). This probably prevents people from running `make test-full`, which `CodingStandards.md` recommends.
There are probably multiple bugs involved. (I can open child tickets for them if people think that's bettter.) Some of them involve invoking `run_tests.py` with a working directory not at the top of the stem source tree. At least one other involves the lack of a `tor` binary in the default `$PATH`. (I'm not sure whether the test in question expects to run the system `tor`.)
Running
```
STEM_SOURCE_DIR=`pwd`/../stem CHUTNEY_PATH=`pwd`/../chutney make test-stem
```
produces many errors like
```
Starting ./src/or/tor...
failed to start tor: './src/or/tor' doesn't exist
```
This suggests that `run_tests.py` doesn't cope well with a relative path to the `tor` binary, so as a workaround I tried setting `TESTING_TOR_BINARY` like
```
STEM_SOURCE_DIR=`pwd`/../stem CHUTNEY_PATH=`pwd`/../chutney make TESTING_TOR_BINARY=`pwd`/src/or/tor test-stem
```
which produced
```
installation... failed (0.04s)
test_install 37 ms [SUCCESS]
test_sdist [FAILURE]
======================================================================
FAIL: test_sdist
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/tlyu/src/stem/test/util.py", line 245, in wrapped
return func(self, *args, **kwargs)
File "/home/tlyu/src/stem/test/integ/installation.py", line 144, in test_sdist
raise SDIST_FAILURE
AssertionError: /home/tlyu/src/stem/dist already exists, maybe you manually ran 'python setup.py sdist'?
----------------------------------------------------------------------
Ran 2 tests in 0.038s
FAILED (failures=1)
```
and
```
process... failed (1.03s)
test_can_run_multithreaded (already ran) [SKIPPED]
test_dump_config_argument 182 ms [SUCCESS]
test_hash_password 42 ms [SUCCESS]
test_hash_password_requires_argument 33 ms [SUCCESS]
test_help_argument 70 ms [SUCCESS]
test_hush_argument 70 ms [SUCCESS]
test_launch_tor_with_config_via_file (already ran) [SKIPPED]
test_launch_tor_with_config_via_stdin (already ran) [SKIPPED]
test_launch_tor_with_timeout (already ran) [SKIPPED]
test_list_fingerprint_argument 230 ms [SUCCESS]
test_list_torrc_options_argument 34 ms [SUCCESS]
test_no_orphaned_process [FAILURE]
test_quiet_argument 37 ms [SUCCESS]
test_take_ownership_via_controller (already ran) [SKIPPED]
test_take_ownership_via_pid (already ran) [SKIPPED]
test_torrc_arguments 46 ms [SUCCESS]
test_torrc_arguments_via_stdin 45 ms [SUCCESS]
test_validate_config_argument 95 ms [SUCCESS]
test_version_argument 39 ms [SUCCESS]
test_with_invalid_config (already ran) [SKIPPED]
test_with_missing_torrc 97 ms [SUCCESS]
======================================================================
ERROR: test_no_orphaned_process
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/tlyu/src/stem/test/util.py", line 260, in wrapped
return func(self, *args, **kwargs)
File "/usr/lib/python2.7/dist-packages/mock/mock.py", line 1305, in patched
return func(*args, **keywargs)
File "/home/tlyu/src/stem/test/integ/process.py", line 217, in test_no_orphaned_process
stem.process.launch_tor()
File "/home/tlyu/src/stem/stem/process.py", line 98, in launch_tor
raise OSError("'%s' isn't available on your system. Maybe it's not in your PATH?" % tor_cmd)
OSError: 'tor' isn't available on your system. Maybe it's not in your PATH?
----------------------------------------------------------------------
Ran 21 tests in 1.033s
FAILED (errors=1, skipped=7)
```
The summary says
```
TESTING FAILED (72 seconds)
[RUN_NONE] test_sdist (test.integ.installation.TestInstallation) ... FAIL
[RUN_NONE] test_no_orphaned_process (test.integ.process.TestProcess) ... ERROR
[RUN_OPEN] test_no_orphaned_process (test.integ.process.TestProcess) ... ERROR
[RUN_PASSWORD] test_no_orphaned_process (test.integ.process.TestProcess) ... ERROR
[RUN_COOKIE] test_no_orphaned_process (test.integ.process.TestProcess) ... ERROR
[RUN_MULTIPLE] test_no_orphaned_process (test.integ.process.TestProcess) ... ERROR
[RUN_SOCKET] test_no_orphaned_process (test.integ.process.TestProcess) ... ERROR
[RUN_SCOOKIE] test_no_orphaned_process (test.integ.process.TestProcess) ... ERROR
[RUN_PTRACE] test_no_orphaned_process (test.integ.process.TestProcess) ... ERROR
You can re-run just these tests with:
/home/tlyu/src/tor/../stem/run_tests.py --tor /home/tlyu/src/tor/src/or/tor --all --log notice --target RUN_ALL --test test.integ.installation
/home/tlyu/src/tor/../stem/run_tests.py --tor /home/tlyu/src/tor/src/or/tor --all --log notice --target RUN_ALL --test test.integ.process
```
Running from the top of the stem source tree
```
./run_tests.py --tor `pwd`/../tor/src/or/tor --all --log notice --target RUN_ALL --test test.integ.installation
```
succeeds
```
14 TESTS WERE SKIPPED
TESTING PASSED (10 seconds)
```
but
```
./run_tests.py --tor `pwd`/../tor/src/or/tor --all --log notice --target RUN_ALL --test test.integ.process
```
still fails in the same way. I don't have a system `tor` binary installed in this VM, but I don't see any documentation that such is required for this test. Also it would be good to not accidentally test the system `tor` binary when we intend to test a specific other one.Tor: 0.3.2.x-finalDamian JohnsonDamian Johnsonhttps://gitlab.torproject.org/legacy/trac/-/issues/10411Downloadable tutorial examples2020-06-13T16:49:49ZDamian JohnsonDownloadable tutorial examplesMarkus makes the good point that it would be nice for visitors to be able to download example.py source files for our tutorial examples. The trick with this will be to avoid duplicating the code. Ideally, we'd have a directory with tutor...Markus makes the good point that it would be nice for visitors to be able to download example.py source files for our tutorial examples. The trick with this will be to avoid duplicating the code. Ideally, we'd have a directory with tutorial examples and they'd be injected into the ReST as part of the build process.Damian JohnsonDamian Johnsonhttps://gitlab.torproject.org/legacy/trac/-/issues/15349Stem out of date in deb.torproject.org (1.2.2 rather than 1.3.0)2020-06-13T16:49:27ZTracStem out of date in deb.torproject.org (1.2.2 rather than 1.3.0)Running the script is giving the error:
File "cursesBW.py", line 68, in _handle_bandwidth_event _render_graph(window, bandwidth_rates)
File "cursesBW.py", line 78, in ...Running the script is giving the error:
File "cursesBW.py", line 68, in _handle_bandwidth_event _render_graph(window, bandwidth_rates)
File "cursesBW.py", line 78, in _render_graph
label = "Downloaded (%s/s):" % str_tools.size_label(download_rates[0], 1)
AttributeError: 'module' object has no attribute 'size_label'
After seeking, I found the symbol definition of 'get_size_label' in:
/usr/lib/python2.7/dist-packages/stem/util/str_tools.py
I substituted 'size_label' by 'get_size_label' and it worked.
**Trac**:
**Username**: Lluis SalaDamian JohnsonDamian Johnsonhttps://gitlab.torproject.org/legacy/trac/-/issues/9103Allow ignoring certain consensus-health warnings2020-06-13T16:47:12ZKarsten LoesingAllow ignoring certain consensus-health warningsThe consensus-health script produces some rather persistent warnings. For example, here's the most recent list of warnings:
```
WARNING: The following directory authorities are not reporting bandwidth scanner results: turtles
NOTICE: T...The consensus-health script produces some rather persistent warnings. For example, here's the most recent list of warnings:
```
WARNING: The following directory authorities are not reporting bandwidth scanner results: turtles
NOTICE: The certificates of the following directory authorities expire within the next two months: Faravahar 2013-08-09 03:46:54, maatuska 2013-08-06 07:58:18
NOTICE: The following directory authorities recommend other client versions than the consensus: moria1 -0.2.4.14-alpha
NOTICE: The following directory authorities recommend other server versions than the consensus: moria1 -0.2.4.14-alpha
```
People who watch the consensus-health list are well aware of these warnings and know that two certs expire in 1.5 months and that Roger has left Internet land for at least this week. That leaves just the first issue that somebody should get Mike to resolve.
The current approach to reduce noise is as follows: if there are no new warnings or if a warning persists for a given number of hours, don't send an email.
There should be an option to ignore certain warnings until a given timestamp. For example, I'd want to ignore expiring certs until mid-July and moria1 recommending different versions until next Monday. I'd like to edit a text file on yatei that contains the warning text and a timestamp until when to ignore this warning.
This list of ignored warnings should also be added to the consensus-health.html page, so that everyone can look up which warnings are currently ignored. The text file could also contain a comment saying why the warning is ignored and decide if that's a good idea or not.
Hopefully, this reduces noise even more and makes authority operators pay more attention to the consensus-health list again.Damian JohnsonDamian Johnsonhttps://gitlab.torproject.org/legacy/trac/-/issues/8797Consensus-health checker should warn if auths assign flags to unusually few o...2020-06-13T16:47:12ZKarsten LoesingConsensus-health checker should warn if auths assign flags to unusually few or many relaysWe should notice cases like #8683 much earlier by adding an alert to the consensus-health checker. It could simply compare the number of flags in a vote and in the consensus. If there are more than, say, 250 relays that have a certain ...We should notice cases like #8683 much earlier by adding an alert to the consensus-health checker. It could simply compare the number of flags in a vote and in the consensus. If there are more than, say, 250 relays that have a certain flag only in a vote or only in the consensus, the consensus-health checker should emit a warning. See also the bottom of the [consensus-health output page](https://metrics.torproject.org/consensus-health.html) for recent numbers. Related to #8164.Damian JohnsonDamian Johnsonhttps://gitlab.torproject.org/legacy/trac/-/issues/32842Give notifications if tor26 is missing from the consensus2020-06-13T16:21:10ZSebastian HahnGive notifications if tor26 is missing from the consensusIt should be possible to infer whether tor26 is down or not by looking at the consensus. Would be nice to have that if it isn't too much work :)It should be possible to infer whether tor26 is down or not by looking at the consensus. Would be nice to have that if it isn't too much work :)Damian JohnsonDamian Johnsonhttps://gitlab.torproject.org/legacy/trac/-/issues/30940Limit fallback directory DocTor IRC list to 5 relays2020-06-13T16:21:09ZteorLimit fallback directory DocTor IRC list to 5 relaysListing all the down fallback directory mirrors doesn't actually add any useful information.
Instead, we could replace the remaining failures with "(N more failures)".
Can we please also put the summary on IRC, it seems to be in the ema...Listing all the down fallback directory mirrors doesn't actually add any useful information.
Instead, we could replace the remaining failures with "(N more failures)".
Can we please also put the summary on IRC, it seems to be in the email, but not on IRC:
```
40/157 (25%) fallback directories have become slow or unresponsive...
```Damian JohnsonDamian Johnsonhttps://gitlab.torproject.org/legacy/trac/-/issues/29094Re-enable the fallback checks with a 25% threshold2020-06-13T16:21:09ZteorRe-enable the fallback checks with a 25% thresholdHi atagar,
We rebuilt the fallback list in #24786.
Please re-enable the fallback checks, and cc phoul on the emails.
ThanksHi atagar,
We rebuilt the fallback list in #24786.
Please re-enable the fallback checks, and cc phoul on the emails.
ThanksDamian JohnsonDamian Johnson