Faravahar is gone, see: tpo/core/tor#40688 for details.

Closes: #40030.

Closes: #40029.

Georg Koppen authored 3 years ago and David Goulet committed 3 years ago

We use the opportunity to reduce noise in #tor-bots by removing
consensus-health checker output. Relevant parties get Cc'ed in doctor's
output or one can subscribe to the tor-consensus-health@ list.

Closes: #40027.

The old IP address is supposed to stop working at the begin of August
according to Alex. No need to check for it anymore.

dannenberg needs to get a handle on the overload situation first.

Damian Johnson authored 4 years ago and Georg Koppen committed 4 years ago

DIRAUTH_SKIP_CHECKS prevented us from downloading (and by extension checking)
a dirauth's consensus documents, but it did not suppress other checks (presence
in votes, ORPort reachability, etc). This expands the suppression to cover
everything that references the DIRECTORY_AUTHORITIES global, which should
hopefully be everything.

------------------------------------------------------------
DocTor Output Before
------------------------------------------------------------

WARNING: moria1 is missing the server descriptor of dannenberg
WARNING: moria1 is missing the server descriptor of Faravahar
WARNING: dizum is missing the server descriptor of dannenberg
WARNING: dizum is missing the server descriptor of Faravahar
WARNING: gabelmoo is missing the server descriptor of dannenberg
WARNING: gabelmoo is missing the server descriptor of Faravahar
WARNING: maatuska is missing the server descriptor of dannenberg
WARNING: maatuska is missing the server descriptor of Faravahar
WARNING: Faravahar is missing the server descriptor of dannenberg
WARNING: longclaw is missing the server descriptor of dannenberg
WARNING: longclaw is missing the server descriptor of Faravahar
WARNING: bastet is missing the server descriptor of dannenberg
WARNING: bastet is missing the server descriptor of Faravahar
WARNING: The following directory authorities do not support the consensus method that the consensus uses: Faravahar
NOTICE: Consensus fetched from dizum was missing the following authority signatures: dannenberg, Faravahar
NOTICE: Consensus fetched from gabelmoo was missing the following authority signatures: dannenberg, Faravahar
NOTICE: Consensus fetched from maatuska was missing the following authority signatures: dannenberg, Faravahar
NOTICE: Consensus fetched from Faravahar was missing the following authority signatures: dannenberg, Faravahar
NOTICE: Consensus fetched from longclaw was missing the following authority signatures: dannenberg, Faravahar
NOTICE: Consensus fetched from bastet was missing the following authority signatures: dannenberg, Faravahar
NOTICE: The following directory authorities are not reporting bandwidth scanner results: Faravahar
WARNING: The following authorities are missing from the consensus: dannenberg, Faravahar
NOTICE: Authorities disagree about the BadExit flag for 867982ADCFBE55EC1C00A5B34C8991FF87605906 (with flag: longclaw, gabelmoo, moria1, without flag: Faravahar)
NOTICE: Authorities disagree about the BadExit flag for B9324F1BEDB1DF62AEE219B0CC41507CE710BA3C (with flag: longclaw, gabelmoo, moria1, without flag: Faravahar)
NOTICE: Authorities disagree about the BadExit flag for 81F59E8936D0B2105182E4B8200AC07D9274961F (with flag: longclaw, gabelmoo, moria1, without flag: Faravahar)
NOTICE: Authorities disagree about the BadExit flag for 8A3606F17B1DDAC4F8BFA0C99E656FFA11CC747D (with flag: longclaw, gabelmoo, moria1, without flag: Faravahar)
NOTICE: Authorities disagree about the BadExit flag for 35449EB3D025CC24601FB43884F9699367D677CF (with flag: longclaw, gabelmoo, without flag: Faravahar, moria1)

------------------------------------------------------------
DocTor Output After
------------------------------------------------------------

WARNING: moria1 is missing the server descriptor of Faravahar
WARNING: dizum is missing the server descriptor of Faravahar
WARNING: gabelmoo is missing the server descriptor of Faravahar
WARNING: maatuska is missing the server descriptor of Faravahar
WARNING: longclaw is missing the server descriptor of Faravahar
WARNING: bastet is missing the server descriptor of Faravahar
WARNING: The following directory authorities do not support the consensus method that the consensus uses: Faravahar
NOTICE: Consensus fetched from moria1 was missing the following authority signatures: Faravahar
NOTICE: Consensus fetched from dizum was missing the following authority signatures: Faravahar
NOTICE: Consensus fetched from gabelmoo was missing the following authority signatures: Faravahar
NOTICE: Consensus fetched from maatuska was missing the following authority signatures: Faravahar
NOTICE: Consensus fetched from Faravahar was missing the following authority signatures: Faravahar
NOTICE: Consensus fetched from longclaw was missing the following authority signatures: Faravahar
NOTICE: Consensus fetched from bastet was missing the following authority signatures: Faravahar
NOTICE: The following directory authorities are not reporting bandwidth scanner results: Faravahar
WARNING: The following authorities are missing from the consensus: Faravahar
NOTICE: Authorities disagree about the BadExit flag for 81F59E8936D0B2105182E4B8200AC07D9274961F (with flag: gabelmoo, moria1, longclaw, without flag: Faravahar)
NOTICE: Authorities disagree about the BadExit flag for 35449EB3D025CC24601FB43884F9699367D677CF (with flag: gabelmoo, longclaw, without flag: Faravahar, moria1)
NOTICE: Authorities disagree about the BadExit flag for B9324F1BEDB1DF62AEE219B0CC41507CE710BA3C (with flag: gabelmoo, moria1, longclaw, without flag: Faravahar)
NOTICE: Authorities disagree about the BadExit flag for 8A3606F17B1DDAC4F8BFA0C99E656FFA11CC747D (with flag: gabelmoo, moria1, longclaw, without flag: Faravahar)
NOTICE: Authorities disagree about the BadExit flag for 867982ADCFBE55EC1C00A5B34C8991FF87605906 (with flag: gabelmoo, moria1, longclaw, without flag: Faravahar)

This got caught by the 2to3 tool.

danneberg temporarily migrated to FreeBSD to fix the LibreSSL issue
(see: tpo/core/tor#40128). Andreas agreed, too, to remove the previous
special treatment for dannenberg which got implemented earlier this
year while the directory authorities were under heavy load.

Until tor#40128 is solved there is not much the operator of dannenberg
can do.

al asked for disabling DocTor checks as he shut down tor temporarily
due to load issues and time to cope with them.

Good catch from starlight on ticket #32842 that these require a comma so the
constants are tuples rather than strings.

The last commit added unused DIRAUTH_SKIP_REACHABLE and DIRAUTH_SKIP_SEEN
constants. If we have use for this at some point then happy to reintroduce
them, but for the moment dead code is unhelpful.

Starlight authored 5 years ago and Damian Johnson committed 5 years ago

replace hard coded exclusion of authority tor26 with three
lists that during checks determine which authorities to not
a) check reachability for
b) check presence in consensus for
c) request and validate documents from

correct defect where router-line IPv4 OR address was not
checked for reachability

eliminate Python 3 exception by truncating array index
expression result to integer (further 2to3 revisions
applied during testing but not committed)

This reverts commit 3f1c5f67 and 7b121dc3.

Dannenberg will be unable to sign the consensus until November due to lack of
the present consensus in Tor's LTS release (and by extension openbsd's).

Dizum moved...

  https://trac.torproject.org/projects/tor/ticket/31406

Alex has requested monitoring of its old address, which will hopefully forward
until clients that reference it get deprecated.

Happy new year! Bumping the dates for 2019...

  % find . -type f -iname '*.py' -exec sed -i 's/-2018/-2019/g' "{}" +;
  % grep -R "# Copyright 2018," ./*

Only moria1 votes on the new flag. Treating it in the same way as BadExit...

  https://lists.torproject.org/pipermail/tor-consensus-health/2018-December/009451.html

Correction thanks to juga and nusenu.

Oops, need to be a tad more defensive here...

	Traceback (most recent call last):
  	File "/srv/doctor.torproject.org/doctor/consensus_health_checker.py", line 976, in <module>
    	main()
	  File "/srv/doctor.torproject.org/doctor/consensus_health_checker.py", line 258, in main
  	  issues += run_checks(consensuses, votes)
	  File "/srv/doctor.torproject.org/doctor/consensus_health_checker.py", line 352, in run_checks
  	  issues = checker(latest_consensus, consensuses, votes)
	  File "/srv/doctor.torproject.org/doctor/consensus_health_checker.py", line 640, in has_authority_flag
  	  seen_authorities.remove('tor26')
	KeyError: 'tor26'

Stem no longer provides this information. Hardcoding it in DocTor.

Using stem's new stem.directory module rather than the deprecated aliases.

Interesting! Roger reported an issue where he isn't getting notified about
moria1's bwauth scanner being down. In taking a look at the logs I'm indeed
seeing something funky...

  04/21/2018 21:05:46 [DEBUG] NOTICE: The following directory authorities are not reporting bandwidth scanner results: gabelmoo
  04/21/2018 21:05:47 [INFO] Suppressing The_following_directory_authorities_are_not_reporting_bandwidth_scanner_results:_gabelmoo, time remaining is 1
  3 hours
  04/21/2018 21:05:47 [INFO] All 1 issues were suppressed. Not sending a notification.
  04/21/2018 21:05:47 [DEBUG] Checks finished, runtime was 46.84 seconds
  04/21/2018 22:05:45 [DEBUG] ERROR: The following directory authorities are not reporting bandwidth scanner results: gabelmoo, moria1
  04/21/2018 22:05:46 [INFO] All 1 issues were suppressed. Not sending a notification.
  04/21/2018 22:05:46 [DEBUG] Checks finished, runtime was 45.56 seconds

I suspect what's happening is this...

  * When only gabelmoo is down this is a NOTICE runlevel notification, which
    works.

  * When moria1 is down as well it's an ERROR runlevel notice which should
    generate an email every hour, but isn't. I think this is due to a bug where
    ERROR notices in specific are getting incorrectly suppressed.

Lets give this a try...

Oops. The message included a float that changes each time, causing us to send a
notice every hour rather than daily as intended for notice runlevel messages.

Roger had the great suggestion that we should check for clock skew...

  https://trac.torproject.org/projects/tor/ticket/25768

Expanded stem a bit along with this to surface response headers.

To cut down on abuse tor26 rejects DirPort requests without a '.z' suffix
(technically it throttles them, but I've never seen one work so effectively
it's blocked). Stem no longer appends this suffix, using headers to indicate
the compression it wants instead.

When four authorites have the BadExit flag and one does not we only need to
notify minority part (and same vice versa). Noticed thanks to Linus.

Stem's integ tests ran into an issue where tor26 lacked moria1's server
descriptor. Turns out this is a longstanding issue due to tor26 holding
on to an outdated ed25519 key. Providing a notice for this.

When an authority first starts up it doesn't vote on the Running flag. Smarter
thing would be to check the known-flags in the vote to see if Running is there,
but for now just dropping the false alarms...

  https://lists.torproject.org/pipermail/tor-consensus-health/2018-February/008580.html

The counts vary every hour so time based suppressions didn't work.

Suppression requested by Roger...

  https://lists.torproject.org/pipermail/tor-dev/2018-February/012943.html

Recently dannenberg stopped issuing the Exit flag. Check Requested by nusenu...

https://lists.torproject.org/pipermail/tor-dev/2018-February/012918.html

Disabling this per Roger's request.

Silly I suppose to do right before new years, but just noticed they still said
2016.

Earlier gabelmoo's bwauth had issues and after a while Sebastian disabled it.
During it we disabled this check so he woulnd't spam for having *or* omitting
bwauth measurements. Sebastian's now running a bwauth again so dropping this
exception.

Requested by Sebastian.

I need to be more careful about code I write before my morning coffee. :P

Stupid me, missing a bracket. Would be nice if we had a test suite for these... :/