This lets us make a lot of other stuff const, allows the compiler to
generate (slightly) better code, and will make me get slightly fewer
patches from folks who stick mutable stuff into or_options_t.

const: because not every input is an output!

This patch introduces a few new functions in router.c to produce a
more helpful description of a node than its nickame, and then tweaks
nearly all log messages taking a nickname as an argument to call these
functions instead.

There are a few cases where I left the old log messages alone: in
these cases, the nickname was that of an authority (whose nicknames
are useful and unique), or the message already included an identity
and/or an address.  I might have missed a couple more too.

This is a fix for bug 3045.

Previously, if they changed in torrc during a SIGHUP, all was well,
since we would just clear all transient entries from the addrmap
thanks to bug 1345.  But if you changed them from the controller, Tor
would leave old mappings in place.

The VirtualAddrNetwork bug has been here since 0.1.1.19-rc; the
AutomapHosts* bug has been here since 0.2.0.1-alpha.

This bug couldn't happen when TrackExitHosts changed in torrc, since
the SIGHUP to reload the torrc would clear out all the transient
addressmap entries before.  But if you used SETCONF to change
TrackExitHosts, old entries would be left alone: that's a bug, and so
this is a bugfix on Tor 0.1.0.1-rc.

Here I looked at the results of the automated conversion and cleaned
them up as follows:

   If there was a tor_memcmp or tor_memeq that was in fact "safe"[*] I
   changed it to a fast_memcmp or fast_memeq.

   Otherwise if there was a tor_memcmp that could turn into a
   tor_memneq or tor_memeq, I converted it.

This wants close attention.

[*] I'm erring on the side of caution here, and leaving some things
as tor_memcmp that could in my opinion use the data-dependent
fast_memcmp variant.

This commit is _exactly_ the result of

perl -i -pe 's/\bmemcmp\(/tor_memcmp\(/g' src/*/*.[ch]
perl -i -pe 's/\!\s*tor_memcmp\(/tor_memeq\(/g' src/*/*.[ch]
perl -i -pe 's/0\s*==\s*tor_memcmp\(/tor_memeq\(/g' src/*/*.[ch]
perl -i -pe 's/0\s*!=\s*tor_memcmp\(/tor_memneq\(/g' src/*/*.[ch]
git checkout src/common/di_ops.[ch]
git checkout src/or/test.c
git checkout src/common/test.h

Previously, it would remove every trackhostexits-derived mapping
*from* xyz.<exitname>.exit; it was supposed to remove every
trackhostexits-derived mapping *to* xyz.<exitname>.exit.

Bugfix on 0.2.0.20-rc: fixes an XXX020 added while staring at bug-1090
issues.

Roger Dingledine authored Apr 26, 2011 and Nick Mathewson committed Apr 27, 2011

Now we believe it to be the case that we never build a circuit for our
stream that has an unsuitable exit, so we'll never need to use such
a circuit. The risk is that we have some code that builds the circuit,
but now we refuse to use it, meaning we just build a bazillion circuits
and ignore them all.

IOW, if we were using TrackExitHosts, and we added an excluded node or
removed a node from exitnodes, we wouldn't actually remove the mapping
that points us at the new node.

Also, note with an XXX022 comment a place that I think we are looking
at the wrong string.

ExcludeExitNodes foo now means that foo.exit doesn't work.  If
StrictNodes is set, then ExcludeNodes foo also overrides foo.exit.

foo.exit , however, still works even if foo is not listed in ExitNodes.

Roger Dingledine authored Mar 11, 2011 and Nick Mathewson committed Apr 26, 2011

add a note reminding us to do this for foo.moria1.exit if we decide to.

Since svn r1475/git 5b6099e8 in tor-0.0.6, we have responded to an
exhaustion of all 65535 stream IDs on a circuit by marking that
circuit for close.  That's not the right response.  Instead, we
should mark the circuit as "too dirty for new circuits".

Of course in reality this isn't really right either.  If somebody
has managed to cram 65535 streams onto a circuit, the circuit is
probably not going to work well for any of those streams, so maybe
we should be limiting the number of streams on an origin circuit
concurrently.

Also, closing the stream in this case is probably the wrong thing to
do as well, but fixing that can also wait.

In afe414 (tor-0.1.0.1-rc~173), when we moved to
connection_edge_end_errno(), we used it in handling errors from
connection_connect().  That's not so good, since by the time
connection_connect() returns, the socket is no longer set, and we're
supposed to be looking at the socket_errno return value from
connection_connect() instead.  So do what we should've done, and
look at the socket_errno value that we get from connection_connect().

Remove some, postpone others, leave some alone.  Now the only
remaining XXX022s are ones that seem important to fix or investigate.

Right now, we only consider sending stream-level SENDME cells when we
have completely flushed a connection_edge's outbuf, or when it sends
us a DATA cell.  Neither of these is ideal for throughput.

This patch changes the behavior so we now call
connection_edge_consider_sending_sendme when we flush _some_ data from
an edge outbuf.

Fix for bug 2756; bugfix on svn r152.

Suggested by rransom.  Probably necessary for testing network mode.

We detect and reject said attempts if there is no chosen exit node or
circuit: connecting to a private addr via a randomly chosen exit node
will usually fail (if all exits reject private addresses), is always
ill-defined (you're not asking for any particular host or service),
and usually an error (you've configured all requests to go over Tor
when you really wanted to configure all _remote_ requests to go over
Tor).

This can also help detect forwarding loop requests.

Found as part of bug2279.

Found by cypherpunks; fixes more of 2328.  Bug was introduced in 3623a122;
first appeared in 0.2.0.5-alpha.

Partial revert of 22f723e4.

Bugfix on 0.2.3.0-alpha

We were not decrementing "available" every time we did
++next_virtual_addr in addressmap_get_virtual_address: we left out the
--available when we skipped .00 and .255 addresses.

This didn't actually cause a bug in most cases, since the failure mode
was to keep looping around the virtual addresses until we found one,
or until available hit zero.  It could have given you an infinite loop
rather than a useful message, however, if you said "VirtualAddrNetwork
127.0.0.255/32" or something broken like that.

Spotted by cypherpunks

Fix for bug 2328; bugfix on 0.1.2.1-alpha; bug found by doorss.

We were decrementing "available" twice for each in-use address we ran
across.  This would make us declare that we ran out of virtual
addresses when the address space was only half full.

This will avoid some signed/unsigned assignment-related bugs.

Still to go: some pipes, all stdio files.

Robert Hogan authored Oct 17, 2010 and Nick Mathewson committed Nov 12, 2010

https://trac.torproject.org/projects/tor/ticket/1859

Use router_get_by_digest() instead of router_get_by_hexdigest()
in circuit_discard_optional_exit_enclaves() and
rend_client_get_random_intro(), per Nick's comments.

Using router_get_by_digest() in rend_client_get_random_intro() will
break hidden services published by Tor versions pre 0.1.2.18 and
0.2.07-alpha as they only publish by nickname. This is acceptable
however as these versions only publish to authority tor26 and
don't work for versions in the 0.2.2.x series anyway.

Robert Hogan authored Sep 30, 2010 and Nick Mathewson committed Nov 12, 2010

https://trac.torproject.org/projects/tor/ticket/1859

There are two problems in this bug:

1. When an OP makes a .exit request specifying itself as the exit, and the exit
   is not yet listed, Tor gets all the routerinfos needed for the circuit but
   discovers in circuit_is_acceptable() that its own routerinfo is not in the
   routerdigest list and cannot be used. Tor then gets locked in a cycle of
   repeating these two steps. When gathering the routerinfos for a circuit,
   specifically when the exit has been chosen by .exit notation, Tor needs to
   apply the same rules it uses later on when deciding if it can build a
   circuit with those routerinfos.

2. A different bug arises in the above situation when the Tor instance's
   routerinfo *is* listed in the routerlist, it shares its nickname with a
   number of other Tor nodes, and it does not have 'Named' rights to its
   nickname.
   So for example, if (i) there are five nodes named Bob in the network, (ii) I
   am running one of them but am flagged as 'Unnamed' because someone else
   claimed the 'Bob' nickname first, and (iii) I run my Tor as both client
   and exit the following can happen to me:
     - I go to www.evil.com
     - I click on a link www.evil.com.bob.exit
     - My request will exit through my own Tor node rather than the 'Named'
       node Bob or any of the others.
     - www.evil.com now knows I am actually browsing from the same computer
       that is running my 'Bob' node

So to solve both issues we need to ensure:

- When fulfilling a .exit request we only choose a routerinfo if it exists in
  the routerlist, even when that routerinfo is ours.
- When getting a router by nickname we only return our own router information
  if it is not going to be used for building a circuit.

We ensure this by removing the special treatment afforded our own router in
router_get_by_nickname(). This means the function will only return the
routerinfo of our own router if it is in the routerlist built from authority
info and has a unique nickname or is bound to a non-unique nickname.

There are some uses of router_get_by_nickname() where we are looking for the
router by name because of a configuration directive, specifically local
declaration of NodeFamilies and EntryNodes and other routers' declaration of
MyFamily. In these cases it is not at first clear if we need to continue
returning our own routerinfo even if our router is not listed and/or has a
non-unique nickname with the Unnamed flag.

The patch treats each of these cases as follows:

Other Routers' Declaration of MyFamily
 This happens in routerlist_add_family(). If another router declares our router
 in its family and our router has the Unnamed flag or is not in the routerlist
 yet, should we take advantage of the fact that we know our own routerinfo to
 add us in anyway? This patch says 'no, treat our own router just like any
 other'. This is a safe choice because it ensures our client has the same view
 of the network as other clients. We also have no good way of knowing if our
 router is Named or not independently of the authorities, so we have to rely on
 them in this.

Local declaration of NodeFamilies
 Again, we have no way of knowing if the declaration 'NodeFamilies
 Bob,Alice,Ringo' refers to our router Bob or the Named router Bob, so we have
to defer to the authorities and treat our own router like any other.

Local declaration of NodeFamilies
 Again, same as above. There's also no good reason we would want our client to
 choose it's own router as an entry guard if it does not meet the requirements
 expected of any other router on the network.

In order to reduce the possibility of error, the patch also replaces two
instances where we were using router_get_by_nickname() with calls to
router_get_by_hexdigest() where the identity digest of the router
is available.

Breaking this out of the last commit because this might be more
controversial.

https://trac.torproject.org/projects/tor/ticket/1859

Use router_get_by_digest() instead of router_get_by_hexdigest()
in circuit_discard_optional_exit_enclaves() and
rend_client_get_random_intro(), per Nick's comments.

Using router_get_by_digest() in rend_client_get_random_intro() will
break hidden services published by Tor versions pre 0.1.2.18 and
0.2.07-alpha as they only publish by nickname. This is acceptable
however as these versions only publish to authority tor26 and
don't work for versions in the 0.2.2.x series anyway.

https://trac.torproject.org/projects/tor/ticket/1859

There are two problems in this bug:

1. When an OP makes a .exit request specifying itself as the exit, and the exit
   is not yet listed, Tor gets all the routerinfos needed for the circuit but
   discovers in circuit_is_acceptable() that its own routerinfo is not in the
   routerdigest list and cannot be used. Tor then gets locked in a cycle of
   repeating these two steps. When gathering the routerinfos for a circuit,
   specifically when the exit has been chosen by .exit notation, Tor needs to
   apply the same rules it uses later on when deciding if it can build a
   circuit with those routerinfos.

2. A different bug arises in the above situation when the Tor instance's
   routerinfo *is* listed in the routerlist, it shares its nickname with a
   number of other Tor nodes, and it does not have 'Named' rights to its
   nickname.
   So for example, if (i) there are five nodes named Bob in the network, (ii) I
   am running one of them but am flagged as 'Unnamed' because someone else
   claimed the 'Bob' nickname first, and (iii) I run my Tor as both client
   and exit the following can happen to me:
     - I go to www.evil.com
     - I click on a link www.evil.com.bob.exit
     - My request will exit through my own Tor node rather than the 'Named'
       node Bob or any of the others.
     - www.evil.com now knows I am actually browsing from the same computer
       that is running my 'Bob' node

So to solve both issues we need to ensure:

- When fulfilling a .exit request we only choose a routerinfo if it exists in
  the routerlist, even when that routerinfo is ours.
- When getting a router by nickname we only return our own router information
  if it is not going to be used for building a circuit.

We ensure this by removing the special treatment afforded our own router in
router_get_by_nickname(). This means the function will only return the
routerinfo of our own router if it is in the routerlist built from authority
info and has a unique nickname or is bound to a non-unique nickname.

There are some uses of router_get_by_nickname() where we are looking for the
router by name because of a configuration directive, specifically local
declaration of NodeFamilies and EntryNodes and other routers' declaration of
MyFamily. In these cases it is not at first clear if we need to continue
returning our own routerinfo even if our router is not listed and/or has a
non-unique nickname with the Unnamed flag.

The patch treats each of these cases as follows:

Other Routers' Declaration of MyFamily
 This happens in routerlist_add_family(). If another router declares our router
 in its family and our router has the Unnamed flag or is not in the routerlist
 yet, should we take advantage of the fact that we know our own routerinfo to
 add us in anyway? This patch says 'no, treat our own router just like any
 other'. This is a safe choice because it ensures our client has the same view
 of the network as other clients. We also have no good way of knowing if our
 router is Named or not independently of the authorities, so we have to rely on
 them in this.

Local declaration of NodeFamilies
 Again, we have no way of knowing if the declaration 'NodeFamilies
 Bob,Alice,Ringo' refers to our router Bob or the Named router Bob, so we have
to defer to the authorities and treat our own router like any other.

Local declaration of NodeFamilies
 Again, same as above. There's also no good reason we would want our client to
 choose it's own router as an entry guard if it does not meet the requirements
 expected of any other router on the network.

In order to reduce the possibility of error, the patch also replaces two
instances where we were using router_get_by_nickname() with calls to
router_get_by_hexdigest() where the identity digest of the router
is available.

but don't refactor the ones that look messy

A node_t is an abstraction over routerstatus_t, routerinfo_t, and
microdesc_t.  It should try to present a consistent interface to all
of them.  There should be a node_t for a server whenever there is
  * A routerinfo_t for it in the routerlist
  * A routerstatus_t in the current_consensus.
(note that a microdesc_t alone isn't enough to make a node_t exist,
since microdescriptors aren't usable on their own.)

There are three ways to get a node_t right now: looking it up by ID,
looking it up by nickname, and iterating over the whole list of
microdescriptors.

All (or nearly all) functions that are supposed to return "a router"
-- especially those used in building connections and circuits --
should return a node_t, not a routerinfo_t or a routerstatus_t.

A node_t should hold all the *mutable* flags about a node.  This
patch moves the is_foo flags from routerinfo_t into node_t.  The
flags in routerstatus_t remain, but they get set from the consensus
and should not change.

Some other highlights of this patch are:

  * Looking up routerinfo and routerstatus by nickname is now
    unified and based on the "look up a node by nickname" function.
    This tries to look only at the values from current consensus,
    and not get confused by the routerinfo_t->is_named flag, which
    could get set for other weird reasons.  This changes the
    behavior of how authorities (when acting as clients) deal with
    nodes that have been listed by nickname.

  * I tried not to artificially increase the size of the diff here
    by moving functions around.  As a result, some functions that
    now operate on nodes are now in the wrong file -- they should
    get moved to nodelist.c once this refactoring settles down.
    This moving should happen as part of a patch that moves
    functions AND NOTHING ELSE.

  * Some old code is now left around inside #if 0/1 blocks, and
    should get removed once I've verified that I don't want it
    sitting around to see how we used to do things.

There are still some unimplemented functions: these are flagged
with "UNIMPLEMENTED_NODELIST()."  I'll work on filling in the
implementation here, piece by piece.

I wish this patch could have been smaller, but there did not seem to
be any piece of it that was independent from the rest.  Moving flags
forces many functions that once returned routerinfo_t * to return
node_t *, which forces their friends to change, and so on.