Tor v0.2.9.1-alpha-dev (git-b3f43a22ab921ce6) - failing options/validate__transproxy test on NetBSD

changed milestone to %Tor: 0.2.9.x-final

added component::core tor/tor milestone::Tor: 0.2.9.x-final netbsd priority::medium regression reporter::yancm resolution::fixed self severity::normal status::closed test type::defect version::tor 0.2.9.1-alpha labels

Do you know how to use "git bisect" ? If so, it would be really helpful to know when this bug first appeared.

Trac:
Milestone: Tor: unspecified to Tor: 0.2.9.x-final
Keywords: self test deleted, self test regression netbsd added

I don't have a NetBSD machine available, but I'm guessing it has something to do with #19449 (moved). Maybe the change enabled transparent proxying by default on NetBSD where it was previously disabled?

Looking at the test (https://gitweb.torproject.org/tor.git/tree/src/test/test_options.c?id=a3d419634bef0bbac4118d08272d59c9fe66a1bb#n1082) there isn't a case that tests NetBSD leading to the failed assertion.

I'll submit a patch that cleans up that part of the code to improve readability and fix the incorrect preprocessor variable for detecting Linux platforms (linux instead of __linux__). Because i don't have a NetBSD machine I'm unsure which firewall system is used on NetBSD. It seems the rest of the Tor code assumes NetBSD is similar to OpenBSD...so i guess it should use pf-divert?

Trac:
0001-Use-the-correct-preprocessor-macro-for-Linux.patch

Replying to nickm:

Do you know how to use "git bisect" ? If so, it would be really helpful to know when this bug first appeared.

Sorry, I do not. I also got out of my habit of daily builds so cannot help that way either.

Trac:
Username: yancm

Replying to cypherpunks:

I don't have a NetBSD machine available, [...] Because i don't have a NetBSD machine I'm unsure which firewall system is used on NetBSD. It seems the rest of the Tor code assumes NetBSD is similar to OpenBSD...so i guess it should use pf-divert?

I'm uncertain, it might be built in. I actually use ipf (pf and npf are also available)

I'll try your patch soon unless it's already been pushed up...

Trac:
Username: yancm

Replying to yancm:

Replying to cypherpunks:

I don't have a NetBSD machine available, [...] Because i don't have a NetBSD machine I'm unsure which firewall system is used on NetBSD. It seems the rest of the Tor code assumes NetBSD is similar to OpenBSD...so i guess it should use pf-divert?

I'm uncertain, it might be built in. I actually use ipf (pf and npf are also available)

I'll try your patch soon unless it's already been pushed up... I don't expect the patch to solve your issue. It was just meant as premature cleanup of related code before actually fixing the issue.

Trac:
Summary: Tor v0.2.9.1-alpha-dev (git-b3f43a22ab921ce6) - failing one test on NetBSD to Tor v0.2.9.1-alpha-dev (git-b3f43a22ab921ce6) - failing options/validate__transproxy test on NetBSD

Hey, does Tor commit ff3e08f2 have anything to do with this? It's about NetBSD and it's right next to a line about transproxy. Maybe there is someplace in the real code that is missing an || defined(__NetBSD__) entry?

I couldn't figure out from the commit whether there's a trac ticket that goes with it though.

The options tests are unfortunately quite fragile, because they depend on the exact log output. (For example, they could be failing simply because the platform name is different.)

It might help to have a unit test log at a useful verbosity level. Try src/test/test --debug or similar.

Trac:
Username: yancm

DebugOutput20161109.rar

test --debug (output 20161110)

Trac:
Username: yancm

I've attached debug output above. I've noticed a couple of other test errors I have not yet reported.

Trac:
Username: yancm

I split off #20634 (moved) for the unnecessary address/get_if_addrs6_list_no_internal failure.

There does seem to be some variation in the results - have you changed versions or applied the attached patch since running the tests?

Because this is what your logs contain:

options/validate__transproxy: [forking] Nov 10 14:47:57.490 [warn] Bad Scheduler
LowWaterMark__ option

  FAIL src/test/test_options.c:1086: assert(!msg)
  [validate__transproxy FAILED]

And this was what you originally reported: {{{ options/validate__transproxy: [forking]

FAIL src/test/test_options.c:1105: assert(tdata)
[validate__transproxy FAILED]

]}}

I believe I had tried applying the patch 3 months back. Since then I have continued tracking the latest tor sources, so the debug output is against the latest git HEAD version - Tor 0.3.0.0-alpha-dev (git-0980787f91cfc420).

Trac:
Username: yancm

Trac:
Description: Latest tor dev sources fail self test.

./src/or/tor -

Aug 22 20:14:56.657 [notice] Tor v0.2.9.1-alpha-dev (git-b3f43a22ab921ce6) running on NetBSD with Libevent 2.1.5-beta, OpenSSL 1.0.2h and Zlib 1.2.3.

NetBSD 6_Stable on i386

1/758 TESTS FAILED. (2 skipped) gmake: * * * [Makefile:7666: test] Error 1

options/validate__transproxy: [forking] FAIL src/test/test_options.c:1105: assert(tdata) [validate__transproxy FAILED]

to

Latest tor dev sources fail self test.

# ./src/or/tor -
Aug 22 20:14:56.657 [notice] Tor v0.2.9.1-alpha-dev (git-b3f43a22ab921ce6) running on NetBSD with Libevent 2.1.5-beta, OpenSSL 1.0.2h and Zlib 1.2.3.

NetBSD 6_Stable on i386

******
 1/758 TESTS FAILED. (2 skipped)
 gmake: * * *  [Makefile:7666: test] Error 1

******
 options/validate__transproxy: [forking]
  FAIL src/test/test_options.c:1105: assert(tdata)
  [validate__transproxy FAILED]

Worth doing in 0.2.9.x if somebody can track down the issue. But it doesn't need to block a release candidate -- especially if we think there's a good chance that it's a bug in our unit tests, rather than a bug in actual Tor code?

Replying to arma:

Worth doing in 0.2.9.x if somebody can track down the issue. But it doesn't need to block a release candidate -- especially if we think there's a good chance that it's a bug in our unit tests, rather than a bug in actual Tor code?

Given it's in an options_validate test, on a non-Linux/macOS/Windows platform, it is almost certainly a bug in the unit tests. It could be as simple as expecting support for a feature on a platform where it does not exist.

However, it could also be a bug in that platform's implementation of the feature.

Either way, no reason to hold up stable.

In e7ade23f9723f55a504ebd8279cbadf0aca108c6 I increased the verborsity of the output of the test, and now on the buildbot I see:

  FAIL src/test/test_options.c:1089: Expected NULL but got 'ipfw is a FreeBSD-specificand OS X/Darwin-specific feature.'

a) "specificand" is a typo.

b) Does netbsd support ipfw or not? We imply "yes" in test_options.c, but "no" in our definition of KERNEL_MAY_SUPPORT_IPFW. Which of these is correct?

I think my branch bug19960_2 is the fix here. I've merged it to master. If it makes the builders happy, it can go into 0.2.9 as well.

Trac:
Status: new to needs_review

Trac:
0001-Use-the-correct-preprocessor-macro-for-Linux.2.patch

I've rebased my patch onto bug19960_2 which addresses the points in comment:2.

Replying to nickm:

b) Does netbsd support ipfw or not? We imply "yes" in test_options.c, but "no" in our definition of KERNEL_MAY_SUPPORT_IPFW. Which of these is correct?

I am pretty sure the answer is no. NetBSD has some support for pf (OpenBSD origins), ipf (independent project? Darren Reed author - no longer under development) and npf which is the new official mechanism in NetBSD as I understand.

I've never used pf, I currently still use ipf only because too lazy to re-visit my rule-sets and replace scripts, etc.

Trac:
Username: yancm

LGTM; merged that too.

Trac:
Resolution: N/A to fixed
Status: needs_review to closed

I've confirmed latest source compile [0.3.0.0-alpha-dev (git-f2445fc608fcb71c)] tests clean. I did not back up and test 0.2.9.0-alpha-dev as I am tracking only active dev source head.

Trac:
Username: yancm

great; thanks for checking!

closed

mentioned in issue #20634 (moved)

moved to tpo/core/tor#19960 (closed)

mentioned in issue tpo/core/tor#20634 (closed)

Tor v0.2.9.1-alpha-dev (git-b3f43a22ab921ce6) - failing options/validate__transproxy test on NetBSD

Child items ...

Activity

./src/or/tor -