Automatc tickets for malformed descriptors
Continuing from https://trac.torproject.org/projects/tor/ticket/27367?replyto=14#comment:14
Replying to atagar:
Related a few past tickets where this has bitten us.
I moved these tickets to #27414 (moved), because relays should do the UTF-8 check before they upload.
But the UTF-8 check won't be enough to catch all parsing errors, so I opened #27415 (moved) so that relays parse descriptors before they upload them.
But the parsing check still won't be enough to catch all spec-implementation divergences. And that's ok. Sometimes we fix the spec, sometimes with fix tor, and sometimes we fix other parsers.
Just a quick note that dirauths could use Stem as a tool for rejecting malformed content. It does stricter validation than the tor binary that descriptors are conformant with the spec. I've been performing this check through DocTor since 2013, filing tickets each time more bad data makes it into the consensus...
https://gitweb.torproject.org/doctor.git/tree/descriptor_checker.py
Bad data chokes not only Stem, but metrics-lib and anything else that ingests it.
Clearly in an ideal world the tor binary itself would do better validation
See the tickets I opened and listed above. I think that's a reasonable amount of validation for the tor binary to do.
In an ideal world, all implementations and the spec would match. When there's a difference, we need to decide what to fix. That's not something we can automate.
but in the absence of that if we took advantage of Stem's validator I wouldn't need to keep filing tickets every few months. Using Stem on dirauths to reject malformed descriptors would prevent these issues upfront, saving Karsten and I hassle.
Running stem as an essential part of dirauth operation is a significant change in tor's security model. It would require a proposal.
It's also a significant change to the consistency of tor's parsing. At the moment, tor uses the same parser and validator for:
- descriptors via authorities
- descriptors direct to bridge clients
- onion service descriptors
If that's a no-go I could also redirect the DocTor check I mention above to email other folks (Nick? teor? Maybe the network team list?) so I don't need to file tickets each time this comes up.
The network team list is administrative and has a closed membership. It's not designed for automatic mailing.
If the checks are safe to make public, then send them to tor-consensus-health, or create another list. (They should be safe, because the validation code is public.)
Whatever you decide, let's adopt the same strategy for the fallback checks. We should extend them beyond just nick and me.
And if there are any other DocTor checks that would benefit from being extended beyond a few people, let's do them at the same time.