Trac issueshttps://gitlab.torproject.org/legacy/trac/-/issues2020-06-13T15:52:55Zhttps://gitlab.torproject.org/legacy/trac/-/issues/33826Add a testing-only option that turns off IPv4 extends2020-06-13T15:52:55ZteorAdd a testing-only option that turns off IPv4 extendsTo test that relays perform IPv6 extends correctly, we should turn off all IPv4 extends in a test network.
The option should be called ExtendAllowIPv4Addresses, and it should be documented and implemented like ExtendAllowIPv6Addresses i...To test that relays perform IPv6 extends correctly, we should turn off all IPv4 extends in a test network.
The option should be called ExtendAllowIPv4Addresses, and it should be documented and implemented like ExtendAllowIPv6Addresses in #33818.
We might also need an ExtendByIPv4ORPort option, similar to ExtendByIPv6ORPort.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/31679Make checkShellScripts.sh handle path errors better2020-06-13T15:45:25ZteorMake checkShellScripts.sh handle path errors bettercheckShellScripts.sh has two minor bugs:
* it should handle being called from its parent directory on systems without realpath
* it should exit with an error status if it can't find the src directory
This is a fix on master, so it doesn...checkShellScripts.sh has two minor bugs:
* it should handle being called from its parent directory on systems without realpath
* it should exit with an error status if it can't find the src directory
This is a fix on master, so it doesn't need a changes file.Tor: 0.4.2.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/30847Failed unit tests with Python 3.82019-06-12T20:41:57ZTracFailed unit tests with Python 3.8There are some errors when running unit tests on Python 3.8:
+ /usr/bin/python3 run_tests.py --unit
======================================================================
INITIALISING ...There are some errors when running unit tests on Python 3.8:
+ /usr/bin/python3 run_tests.py --unit
======================================================================
INITIALISING
======================================================================
stem version... 1.7.1
python version... 3.8.0a4
operating system version... failed
module 'platform' has no attribute 'linux_distribution'
BUILDSTDERR: /builddir/build/BUILD/stem-1.7.1/test/output.py:150: SyntaxWarning: invalid escape sequence \(
BUILDSTDERR: m = re.match('.*( \(test\..*?\)).*', line_content)
BUILDSTDERR: /builddir/build/BUILD/stem-1.7.1/test/output.py:163: SyntaxWarning: invalid escape sequence \.
BUILDSTDERR: m = re.search('(test\.[^)]*)', line_content)
BUILDSTDERR: /builddir/build/BUILD/stem-1.7.1/test/output.py:264: SyntaxWarning: invalid escape sequence \(
BUILDSTDERR: module_match = re.match('.*\((test\.\S+)\.\S+\).*', line_content)
BUILDSTDERR: /builddir/build/BUILD/stem-1.7.1/test/task.py:140: SyntaxWarning: invalid escape sequence \S
BUILDSTDERR: test_match = re.search('^class (\S*)\(unittest.TestCase\):$', file_contents, re.MULTILINE)
BUILDSTDERR: /usr/lib/python3.8/site-packages/pep8.py:110: FutureWarning: Possible nested set at position 1
BUILDSTDERR: EXTRANEOUS_WHITESPACE_REGEX = re.compile(r'[[({] | []}),;:]')
Full log here:
https://copr-be.cloud.fedoraproject.org/results/@python/python3.8/fedora-rawhide-x86_64/00917176-python-stem/build.log.gz
**Trac**:
**Username**: miceliuxDamian JohnsonDamian Johnsonhttps://gitlab.torproject.org/legacy/trac/-/issues/30102For CI testing, put garbage in ~/.torrc2020-06-13T15:40:30ZNick MathewsonFor CI testing, put garbage in ~/.torrcNone of our testing code should use the user's default ~/.torrc file. Let's make sure it doesn't, by making ~/.torrc with garbage in it before running our CI.
See #29702 for an example of this kind of bug.None of our testing code should use the user's default ~/.torrc file. Let's make sure it doesn't, by making ~/.torrc with garbage in it before running our CI.
See #29702 for an example of this kind of bug.Tor: 0.4.2.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/29435coverage script broken by library refactoring2020-06-13T15:38:02ZNick Mathewsoncoverage script broken by library refactoringThe fix is easy here; I'll write a patch.The fix is easy here; I'll write a patch.Tor: 0.3.5.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/28739Add more tests for assigning voting flags in test_voting_flags.c2020-06-13T15:35:16ZNick MathewsonAdd more tests for assigning voting flags in test_voting_flags.cIn test_voting_flags.c we add some tests to make sure that flags are assigned the way we want when authorities make votes. We didn't add tests for every flag, though. We should do that some time.In test_voting_flags.c we add some tests to make sure that flags are assigned the way we want when authorities make votes. We didn't add tests for every flag, though. We should do that some time.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/27854Integration test(s) for tor-resolve2020-06-13T15:32:09Zrl1987Integration test(s) for tor-resolveTor: unspecifiedrl1987rl1987https://gitlab.torproject.org/legacy/trac/-/issues/27717Report tor segfaults when integ testing2018-09-30T19:11:11ZDamian JohnsonReport tor segfaults when integ testingFrom time to time Stem's integration tests crash tor (for example, #27500). When this happens our test output is pretty confusing since from the point of the crash onward we fail with generic 'cannot connect to tor' errors.
There's two ...From time to time Stem's integration tests crash tor (for example, #27500). When this happens our test output is pretty confusing since from the point of the crash onward we fail with generic 'cannot connect to tor' errors.
There's two issues here...
1. Our test output doesn't indicate that tor crashed.
2. When reporting tor segfaults we need its stacktrace, which is swallowed by our tests.
We should adjust our test's get_tor_controller() helper to check if tor is still alive, failing the test with a clearer message if it isn't and providing the tor stacktrace at the end of the test run.Damian JohnsonDamian Johnsonhttps://gitlab.torproject.org/legacy/trac/-/issues/27479Integ failure with ss2019-12-20T23:50:49ZDamian JohnsonInteg failure with ssIntegration test failure spotted by Nick...
```
======================================================================
ERROR: test_connections_by_ss
----------------------------------------------------------------------
Traceback (most ...Integration test failure spotted by Nick...
```
======================================================================
ERROR: test_connections_by_ss
----------------------------------------------------------------------
Traceback (most recent call last):
File "stem/test/integ/util/connection.py", line 49, in test_connections_by_ss
self.check_resolver(Resolver.SS)
File "stem/test/require.py", line 58, in wrapped
return func(self, *args, **kwargs)
File "stem/test/integ/util/connection.py", line 28, in check_resolver
connections = get_connections(resolver, process_pid = runner.get_pid())
File "stem/stem/util/connection.py", line 300, in get_connections
raise IOError('No results found using: %s' % resolver_command)
OSError: No results found using: ss -nptu
----------------------------------------------------------------------
```Damian JohnsonDamian Johnsonhttps://gitlab.torproject.org/legacy/trac/-/issues/27173Unstable unit tests in control.controller2019-02-05T21:15:26ZdmrUnstable unit tests in control.controller#27053 brought about some changes to stem's controller's event handling.
Unfortunately the tests for that are a bit unstable - sometimes failing.
Instead of letting that ticket drag on, the work is being moved to a new ticket for better...#27053 brought about some changes to stem's controller's event handling.
Unfortunately the tests for that are a bit unstable - sometimes failing.
Instead of letting that ticket drag on, the work is being moved to a new ticket for better clarity.
The history behind that is mostly in #27053, so refer there.
A few parts were discussed over IRC, however, so that context will be filled in below...
From [ticket:27053#comment:14]
> [...] The good news is that I figured out the python3 mock issue you found. Turns out there's a difference between PyPI and Python3's mock modules. Fixed...
>
> https://gitweb.torproject.org/stem.git/commit/?id=e75cf25
I confirmed that this works for me.
From [ticket:27053#comment:15] (the latest comment at time of writing):
> Hi Dave, think I got it but tough to be sure. Does this do the trick for you?
>
> https://gitweb.torproject.org/stem.git/commit/?id=abc6f29
Tests are still failing as of this revision, still giving `AssertionError: Expected 'mock' to be called once. Called 0 times.` for py27.
It seems to be more stable, now failing at ~5-10% rate instead of 25-40%. //(Rate not scientific)//Damian JohnsonDamian Johnsonhttps://gitlab.torproject.org/legacy/trac/-/issues/26916Make tox config more useful/friendly for running multiple interpreters/versions2018-07-24T19:33:31ZdmrMake tox config more useful/friendly for running multiple interpreters/versionsRight now `tox` can be a bit of a pain for dev usage.
This ticket is intended to remove some of the hassle for multiple interpreters/versions, and make the default environments config a bit more useful.
I've identified a few improvement...Right now `tox` can be a bit of a pain for dev usage.
This ticket is intended to remove some of the hassle for multiple interpreters/versions, and make the default environments config a bit more useful.
I've identified a few improvements under this theme - these are better described in commits.
**atagar: I've got a PR coming soon**dmrdmrhttps://gitlab.torproject.org/legacy/trac/-/issues/26822Investigate relying on tox's default install capabilities instead of an expli...2019-06-08T18:38:53ZdmrInvestigate relying on tox's default install capabilities instead of an explicit commandFrom [ticket:26811#comment:2]:
> I'm not convinced that the `pip install -e .` line needs to be in the `tox.ini` config //at all//, as I'm pretty sure `tox` will install things just fine without it, but I didn't have the time to explore ...From [ticket:26811#comment:2]:
> I'm not convinced that the `pip install -e .` line needs to be in the `tox.ini` config //at all//, as I'm pretty sure `tox` will install things just fine without it, but I didn't have the time to explore that route further at the present.
>
> So I think the best option moving forward in the interim is to take the patch, and file another ticket to look further into it at a later point. (But let me know your thoughts!)
This ticket is //that// ticket. **Its purpose:** investigate removing the `pip install -e .` line from the config (or note why it might be necessary).Damian JohnsonDamian Johnsonhttps://gitlab.torproject.org/legacy/trac/-/issues/26811tox fails with pip 102018-07-16T21:51:52Zdmrtox fails with pip 10Trying to run `tox` in a fresh `stem` workspace (no `.tox` dir, which contains the virtualenvs) now fails.
Failure output with pip 10:
```
no such option: --allow-all-external
ERROR: InvocationError: 'path/to/pip install --allow-all-ext...Trying to run `tox` in a fresh `stem` workspace (no `.tox` dir, which contains the virtualenvs) now fails.
Failure output with pip 10:
```
no such option: --allow-all-external
ERROR: InvocationError: 'path/to/pip install --allow-all-external -e .'
```
Here's a deprecation note from a tox / pip 9 virtualenv:
```
py27 runtests: commands[0] | pip install --allow-all-external -e .
DEPRECATION: --allow-all-external has been deprecated and will be removed in the future. Due to changes in the repository protocol, it no longer has any effect.
```
With pip 10 (somewhat recently released), this `--allow-all-external` flag seemingly got removed.
**atagar: I'll have a patch for this shortly! **dmrdmrhttps://gitlab.torproject.org/legacy/trac/-/issues/26421Include info about the --quiet / -q option in --help2018-06-24T21:52:16ZdmrInclude info about the --quiet / -q option in --helpI was originally going to file a ticket to //implement// the `--quiet` option, but in looking at the code, saw that it already exists.
It doesn't get displayed when we do `run_tests.py --help`, so make it do that.I was originally going to file a ticket to //implement// the `--quiet` option, but in looking at the code, saw that it already exists.
It doesn't get displayed when we do `run_tests.py --help`, so make it do that.dmrdmrhttps://gitlab.torproject.org/legacy/trac/-/issues/26420Testing - specify literal patterns instead of regex patterns2018-08-06T01:51:01ZdmrTesting - specify literal patterns instead of regex patterns~~w.r.t. [[https://gitweb.torproject.org/stem.git/commit/?id=7711050619af1a2f8ecf4aa87f774baa5f367b3b|7711050619af1a2f8ecf4aa87f774baa5f367b3b]], I was planning to file this ticket anyways, so might as well now for the discussion.~~
ata...~~w.r.t. [[https://gitweb.torproject.org/stem.git/commit/?id=7711050619af1a2f8ecf4aa87f774baa5f367b3b|7711050619af1a2f8ecf4aa87f774baa5f367b3b]], I was planning to file this ticket anyways, so might as well now for the discussion.~~
atagar linked [[StackOverflow answer](https://stackoverflow.com/questions/8672754/how-to-show-the-error-messages-caught-by-assertraises-in-unittest-in-python2-7/8673096#8673096|this)] in the commit message.
(I was a bit behind on filing this ticket, but already started doing the literal `re.escape()` bit in my test cases. Hence atagar's comment in the commit.)
Anyway, here's the ticket text I had started to prep - now //slightly// tweaked:
===
The testing codebase makes pretty extensive use of `unittest.TestCase.assertRaisesRegexp()`.
An example is [[https://gitweb.torproject.org/stem.git/tree/test/integ/client/connection.py?id=0192b29a4784465e5f69f11ced584a54644e4a90#n36|here]]:
```
def test_no_common_link_protocol(self):
"""
Connection without a commonly accepted link protocol version.
"""
for link_protocol in (1, 2, 6, 20):
self.assertRaisesRegexp(stem.SocketError, 'Unable to establish a common link protocol with 127.0.0.1:1113', Relay.connect, '127.0.0.1', test.runner.ORPORT, [link_protocol])
```
The second argument is treated as a regex pattern, so it will actually match more than intended - possibly leading to some false negatives (although unlikely in this example).
The use of `unittest.TestCase.assertRaisesRegexp()` without `re.escape()` for a literal expression is decently common - the use of it intended for a regex is fairly rare (I haven't come across a test yet that I can recall).
Having a "literal" check is possible in (at least) two ways:
```
with self.assertRaises(SomeException) as cm:
do_something(some_param)
self.assertEqual(str(cm.exception), expected_literal)
```
```
self.assertRaisesRegexp(SomeException, '^%s$' % re.escape(expected_literal), do_something, some_param)
```
Importantly, both of these check for //exact// string.
Much of the codebase doesn't use `re.escape()`, and where it does, I didn't see any `^` or `$` (although I didn't search exhaustively), so that could match on substrings, also allowing for subtle bugs.
So we may consider a helper class `StemTestCase(unittest.TestCase)` which adds an `assertRaisesLiteral()` method, to make it easier to do this. (Or some other means of adding that in.)
We could of course take the second option with `re.escape()`, but since a lot of the codebase already doesn't seem to do that, it's probably quite easy to forget, especially the `'^%s$' % ` part.
~~atagar: thoughts on these options? or leave things as-is / `wontfix`?~~
~~(Filing this as a //task// ticket, as it's probably a discussion point more than anything else. I'd expect from the edge cases, there //could// be some defects, some enhancements.)~~
See especially [[comment:2|atagar's comment about implementation thoughts]].dmrdmrhttps://gitlab.torproject.org/legacy/trac/-/issues/26373test_rust.sh should detect when it's being invoked improperly and error out2020-06-13T15:26:40ZTaylor Yutest_rust.sh should detect when it's being invoked improperly and error outWhile attempting to test #26258, I noticed that running src/test/test_rust.sh from the top of the source tree exited with status 0 and no output. It should probably detect that it failed to find any Cargo.toml files and exit with a fail...While attempting to test #26258, I noticed that running src/test/test_rust.sh from the top of the source tree exited with status 0 and no output. It should probably detect that it failed to find any Cargo.toml files and exit with a failure status with an error message. (This seems to happen because some necessary environment variables aren't set.)
Prior to #26258, the find invocation failing would probably have taken care of this, so this change should probably get back ported to the same releases.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/26225Jenkins test failure: test_get_microdescriptors - test may be unreliable2018-06-24T23:10:36ZdmrJenkins test failure: test_get_microdescriptors - test may be unreliableA [[Jenkins test run (`Build #3443 (May 25, 2018 9:40:56 PM) `)](https://jenkins.torproject.org/job/stem-tor-ci/3443/console|recent)] indicates that our `test_get_microdescriptors` test may be a bit unreliable:
```
======================...A [[Jenkins test run (`Build #3443 (May 25, 2018 9:40:56 PM) `)](https://jenkins.torproject.org/job/stem-tor-ci/3443/console|recent)] indicates that our `test_get_microdescriptors` test may be a bit unreliable:
```
======================================================================
21:43:42 ERROR: test_get_microdescriptors
21:43:42 ----------------------------------------------------------------------
21:43:42 Traceback (most recent call last):
21:43:42 File "/srv/jenkins-workspace/workspace/stem-tor-ci/test/require.py", line 58, in wrapped
21:43:42 return func(self, *args, **kwargs)
21:43:42 File "/srv/jenkins-workspace/workspace/stem-tor-ci/test/integ/control/controller.py", line 1164, in test_get_microdescriptors
21:43:42 for desc in controller.get_microdescriptors():
21:43:42 File "/srv/jenkins-workspace/workspace/stem-tor-ci/stem/control.py", line 488, in wrapped
21:43:42 for val in func(self, *args, **kwargs):
21:43:42 File "/srv/jenkins-workspace/workspace/stem-tor-ci/stem/control.py", line 1773, in get_microdescriptors
21:43:42 raise stem.OperationFailed(message = "Data directory doens't contain cached microescriptors (%s)" % cached_descriptor_path)
21:43:42 OperationFailed: Data directory doens't contain cached microescriptors (/srv/jenkins-workspace/workspace/stem-tor-ci/test/data/cached-microdescs)
```
Later test runs have passed, and nothing major has changed in the code.dmrdmrhttps://gitlab.torproject.org/legacy/trac/-/issues/25632Improve stem torrc logging options for integration testing2019-06-10T18:03:19ZdmrImprove stem torrc logging options for integration testingAs suggested by teor in #25631, utilizing the following options in torrc will improve logging for the purpose of integration testing in stem:
* `ProtocolWarnings 1`
* `SafeLogging 0`
* `LogTimeGranularity 1`
The former (`ProtocolWarning...As suggested by teor in #25631, utilizing the following options in torrc will improve logging for the purpose of integration testing in stem:
* `ProtocolWarnings 1`
* `SafeLogging 0`
* `LogTimeGranularity 1`
The former (`ProtocolWarnings 1`) should be used in general with a local test relay (i.e. during development, independent of integration testing), and the log `[warn]`ings surfaced automatically in integration tests, which I believe currently isn't the case. The `[warn]`ings should potentially be treated as test failures when emitted (with exception of tests that directly try to violate the protocol, and check that Tor responds as such).
The latter two will help with debugging when a problem is encountered.Damian JohnsonDamian Johnsonhttps://gitlab.torproject.org/legacy/trac/-/issues/25611Integ test for MAPADDRESS2020-06-13T10:03:39ZDamian JohnsonInteg test for MAPADDRESSStem had an integration test for tor's MAPADDRESS command...
https://gitweb.torproject.org/stem.git/tree/test/integ/control/controller.py#n1120
However, it doesn't work...
```
==========================================================...Stem had an integration test for tor's MAPADDRESS command...
https://gitweb.torproject.org/stem.git/tree/test/integ/control/controller.py#n1120
However, it doesn't work...
```
======================================================================
FAIL: test_mapaddress
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/atagar/Desktop/stem/test/require.py", line 58, in wrapped
return func(self, *args, **kwargs)
File "/home/atagar/Desktop/stem/test/require.py", line 58, in wrapped
return func(self, *args, **kwargs)
File "/home/atagar/Desktop/stem/test/integ/control/controller.py", line 1154, in test_mapaddress
self.assertTrue(stem.util.connection.is_valid_ipv4_address(stem.util.str_tools._to_unicode(ip_addr)), "'%s' isn't an address" % ip_addr)
AssertionError: '<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>403 Forbidden</title>
</head><body>
<h1>Forbidden</h1>
<p>You don't have permission to access /ip
on this server.</p>
</body></html>' isn't an address
```
After several hours of fiddling with it I'm disabling the failing test for now. Things I checked were...
* Tested with both the current stem codebase and last release tag (1.6.0) to rule out a recent regression in stem.
* Tested with both the current tor git codebase and an older version (0.2.7.6) to rule out a regression in tor.
* Through tor browser visited the site this test retrieves (http://ifconfig.me/ip) in case they're blocking tor. No dice.
Open questions are 'why did this test start failing?' and get integ coverage for MAPADDRESS. Happy to outright replace this test with something else that exercises it.Damian JohnsonDamian Johnsonhttps://gitlab.torproject.org/legacy/trac/-/issues/23495Add a chutney network with a HS via a bridge to tor's make test-network-all2020-06-13T15:13:40ZteorAdd a chutney network with a HS via a bridge to tor's make test-network-allNow I've done #22691, let's make bridges+ipv6+hs-v23 part of the default set.Now I've done #22691, let's make bridges+ipv6+hs-v23 part of the default set.Tor: 0.3.2.x-final