I've made an exception for cases where I'm sure that users can't
influence the inputs.  This is likely to cause a slowdown somewhere,
but it's safer to siphash everything and *then* look for cases to
optimize.

This patch doesn't actually get us any _benefit_ from siphash yet,
since we don't really randomize the key at any point.

Also lower maximum interval without directory requests, and raise
maximum download tries.

Implements #6752.

This is an automatically generated commit, from the following perl script,
run with the options "-w -i -p".

  s/smartlist_string_num_isin/smartlist_contains_int_as_string/g;
  s/smartlist_string_isin((?:_case)?)/smartlist_contains_string$1/g;
  s/smartlist_digest_isin/smartlist_contains_digest/g;
  s/smartlist_isin/smartlist_contains/g;
  s/digestset_isin/digestset_contains/g;

In C, we technically aren't supposed to define our own things that
start with an underscore.

This is a purely machine-generated commit.  First, I ran this script
on all the headers in src/{common,or,test,tools/*}/*.h :
==============================

use strict;

my %macros = ();
my %skipped = ();
FILE: for my $fn (@ARGV) {
    my $f = $fn;
    if ($fn !~ /^\.\//) {
	$f = "./$fn";
    }
    $skipped{$fn} = 0;
    open(F, $fn);
    while (<F>) {
	if (/^#ifndef ([A-Za-z0-9_]+)/) {
	    $macros{$fn} = $1;
	    next FILE;
	}
    }
}

print "#!/usr/bin/perl -w -i -p\n\n";
for my $fn (@ARGV) {
    if (! exists $macros{$fn}) {
	print "# No macro known for $fn!\n" if (!$skipped{$fn});
	next;
    }
    if ($macros{$fn} !~ /_H_?$/) {
	print "# Weird macro for $fn...\n";
    }
    my $goodmacro = uc $fn;
    $goodmacro =~ s#.*/##;
    $goodmacro =~ s#[\/\-\.]#_#g;
    print "s/(?<![A-Za-z0-9_])$macros{$fn}(?![A-Za-z0-9_])/TOR_${goodmacro}/g;\n"
}
==============================

It produced the following output, which I then re-ran on those same files:

==============================

s/(?<![A-Za-z0-9_])_TOR_ADDRESS_H(?![A-Za-z0-9_])/TOR_ADDRESS_H/g;
s/(?<![A-Za-z0-9_])_TOR_AES_H(?![A-Za-z0-9_])/TOR_AES_H/g;
s/(?<![A-Za-z0-9_])_TOR_COMPAT_H(?![A-Za-z0-9_])/TOR_COMPAT_H/g;
s/(?<![A-Za-z0-9_])_TOR_COMPAT_LIBEVENT_H(?![A-Za-z0-9_])/TOR_COMPAT_LIBEVENT_H/g;
s/(?<![A-Za-z0-9_])_TOR_CONTAINER_H(?![A-Za-z0-9_])/TOR_CONTAINER_H/g;
s/(?<![A-Za-z0-9_])_TOR_CRYPTO_H(?![A-Za-z0-9_])/TOR_CRYPTO_H/g;
s/(?<![A-Za-z0-9_])TOR_DI_OPS_H(?![A-Za-z0-9_])/TOR_DI_OPS_H/g;
s/(?<![A-Za-z0-9_])_TOR_MEMAREA_H(?![A-Za-z0-9_])/TOR_MEMAREA_H/g;
s/(?<![A-Za-z0-9_])_TOR_MEMPOOL_H(?![A-Za-z0-9_])/TOR_MEMPOOL_H/g;
s/(?<![A-Za-z0-9_])TOR_PROCMON_H(?![A-Za-z0-9_])/TOR_PROCMON_H/g;
s/(?<![A-Za-z0-9_])_TOR_TORGZIP_H(?![A-Za-z0-9_])/TOR_TORGZIP_H/g;
s/(?<![A-Za-z0-9_])_TOR_TORINT_H(?![A-Za-z0-9_])/TOR_TORINT_H/g;
s/(?<![A-Za-z0-9_])_TOR_LOG_H(?![A-Za-z0-9_])/TOR_TORLOG_H/g;
s/(?<![A-Za-z0-9_])_TOR_TORTLS_H(?![A-Za-z0-9_])/TOR_TORTLS_H/g;
s/(?<![A-Za-z0-9_])_TOR_UTIL_H(?![A-Za-z0-9_])/TOR_UTIL_H/g;
s/(?<![A-Za-z0-9_])_TOR_BUFFERS_H(?![A-Za-z0-9_])/TOR_BUFFERS_H/g;
s/(?<![A-Za-z0-9_])_TOR_CHANNEL_H(?![A-Za-z0-9_])/TOR_CHANNEL_H/g;
s/(?<![A-Za-z0-9_])_TOR_CHANNEL_TLS_H(?![A-Za-z0-9_])/TOR_CHANNELTLS_H/g;
s/(?<![A-Za-z0-9_])_TOR_CIRCUITBUILD_H(?![A-Za-z0-9_])/TOR_CIRCUITBUILD_H/g;
s/(?<![A-Za-z0-9_])_TOR_CIRCUITLIST_H(?![A-Za-z0-9_])/TOR_CIRCUITLIST_H/g;
s/(?<![A-Za-z0-9_])_TOR_CIRCUITMUX_EWMA_H(?![A-Za-z0-9_])/TOR_CIRCUITMUX_EWMA_H/g;
s/(?<![A-Za-z0-9_])_TOR_CIRCUITMUX_H(?![A-Za-z0-9_])/TOR_CIRCUITMUX_H/g;
s/(?<![A-Za-z0-9_])_TOR_CIRCUITUSE_H(?![A-Za-z0-9_])/TOR_CIRCUITUSE_H/g;
s/(?<![A-Za-z0-9_])_TOR_COMMAND_H(?![A-Za-z0-9_])/TOR_COMMAND_H/g;
s/(?<![A-Za-z0-9_])_TOR_CONFIG_H(?![A-Za-z0-9_])/TOR_CONFIG_H/g;
s/(?<![A-Za-z0-9_])TOR_CONFPARSE_H(?![A-Za-z0-9_])/TOR_CONFPARSE_H/g;
s/(?<![A-Za-z0-9_])_TOR_CONNECTION_EDGE_H(?![A-Za-z0-9_])/TOR_CONNECTION_EDGE_H/g;
s/(?<![A-Za-z0-9_])_TOR_CONNECTION_H(?![A-Za-z0-9_])/TOR_CONNECTION_H/g;
s/(?<![A-Za-z0-9_])_TOR_CONNECTION_OR_H(?![A-Za-z0-9_])/TOR_CONNECTION_OR_H/g;
s/(?<![A-Za-z0-9_])_TOR_CONTROL_H(?![A-Za-z0-9_])/TOR_CONTROL_H/g;
s/(?<![A-Za-z0-9_])_TOR_CPUWORKER_H(?![A-Za-z0-9_])/TOR_CPUWORKER_H/g;
s/(?<![A-Za-z0-9_])_TOR_DIRECTORY_H(?![A-Za-z0-9_])/TOR_DIRECTORY_H/g;
s/(?<![A-Za-z0-9_])_TOR_DIRSERV_H(?![A-Za-z0-9_])/TOR_DIRSERV_H/g;
s/(?<![A-Za-z0-9_])_TOR_DIRVOTE_H(?![A-Za-z0-9_])/TOR_DIRVOTE_H/g;
s/(?<![A-Za-z0-9_])_TOR_DNS_H(?![A-Za-z0-9_])/TOR_DNS_H/g;
s/(?<![A-Za-z0-9_])_TOR_DNSSERV_H(?![A-Za-z0-9_])/TOR_DNSSERV_H/g;
s/(?<![A-Za-z0-9_])TOR_EVENTDNS_TOR_H(?![A-Za-z0-9_])/TOR_EVENTDNS_TOR_H/g;
s/(?<![A-Za-z0-9_])_TOR_GEOIP_H(?![A-Za-z0-9_])/TOR_GEOIP_H/g;
s/(?<![A-Za-z0-9_])_TOR_HIBERNATE_H(?![A-Za-z0-9_])/TOR_HIBERNATE_H/g;
s/(?<![A-Za-z0-9_])_TOR_MAIN_H(?![A-Za-z0-9_])/TOR_MAIN_H/g;
s/(?<![A-Za-z0-9_])_TOR_MICRODESC_H(?![A-Za-z0-9_])/TOR_MICRODESC_H/g;
s/(?<![A-Za-z0-9_])_TOR_NETWORKSTATUS_H(?![A-Za-z0-9_])/TOR_NETWORKSTATUS_H/g;
s/(?<![A-Za-z0-9_])_TOR_NODELIST_H(?![A-Za-z0-9_])/TOR_NODELIST_H/g;
s/(?<![A-Za-z0-9_])_TOR_NTMAIN_H(?![A-Za-z0-9_])/TOR_NTMAIN_H/g;
s/(?<![A-Za-z0-9_])_TOR_ONION_H(?![A-Za-z0-9_])/TOR_ONION_H/g;
s/(?<![A-Za-z0-9_])_TOR_OR_H(?![A-Za-z0-9_])/TOR_OR_H/g;
s/(?<![A-Za-z0-9_])_TOR_POLICIES_H(?![A-Za-z0-9_])/TOR_POLICIES_H/g;
s/(?<![A-Za-z0-9_])_TOR_REASONS_H(?![A-Za-z0-9_])/TOR_REASONS_H/g;
s/(?<![A-Za-z0-9_])_TOR_RELAY_H(?![A-Za-z0-9_])/TOR_RELAY_H/g;
s/(?<![A-Za-z0-9_])_TOR_RENDCLIENT_H(?![A-Za-z0-9_])/TOR_RENDCLIENT_H/g;
s/(?<![A-Za-z0-9_])_TOR_RENDCOMMON_H(?![A-Za-z0-9_])/TOR_RENDCOMMON_H/g;
s/(?<![A-Za-z0-9_])_TOR_RENDMID_H(?![A-Za-z0-9_])/TOR_RENDMID_H/g;
s/(?<![A-Za-z0-9_])_TOR_RENDSERVICE_H(?![A-Za-z0-9_])/TOR_RENDSERVICE_H/g;
s/(?<![A-Za-z0-9_])_TOR_REPHIST_H(?![A-Za-z0-9_])/TOR_REPHIST_H/g;
s/(?<![A-Za-z0-9_])_TOR_REPLAYCACHE_H(?![A-Za-z0-9_])/TOR_REPLAYCACHE_H/g;
s/(?<![A-Za-z0-9_])_TOR_ROUTER_H(?![A-Za-z0-9_])/TOR_ROUTER_H/g;
s/(?<![A-Za-z0-9_])_TOR_ROUTERLIST_H(?![A-Za-z0-9_])/TOR_ROUTERLIST_H/g;
s/(?<![A-Za-z0-9_])_TOR_ROUTERPARSE_H(?![A-Za-z0-9_])/TOR_ROUTERPARSE_H/g;
s/(?<![A-Za-z0-9_])TOR_ROUTERSET_H(?![A-Za-z0-9_])/TOR_ROUTERSET_H/g;
s/(?<![A-Za-z0-9_])TOR_STATEFILE_H(?![A-Za-z0-9_])/TOR_STATEFILE_H/g;
s/(?<![A-Za-z0-9_])_TOR_STATUS_H(?![A-Za-z0-9_])/TOR_STATUS_H/g;
s/(?<![A-Za-z0-9_])TOR_TRANSPORTS_H(?![A-Za-z0-9_])/TOR_TRANSPORTS_H/g;
s/(?<![A-Za-z0-9_])_TOR_TEST_H(?![A-Za-z0-9_])/TOR_TEST_H/g;
s/(?<![A-Za-z0-9_])_TOR_FW_HELPER_H(?![A-Za-z0-9_])/TOR_TOR_FW_HELPER_H/g;
s/(?<![A-Za-z0-9_])_TOR_FW_HELPER_NATPMP_H(?![A-Za-z0-9_])/TOR_TOR_FW_HELPER_NATPMP_H/g;
s/(?<![A-Za-z0-9_])_TOR_FW_HELPER_UPNP_H(?![A-Za-z0-9_])/TOR_TOR_FW_HELPER_UPNP_H/g;
==============================

Fixes bug 4893.

These changes are pure mechanical, and were generated with this
perl script:

  /usr/bin/perl -w -i.bak -p

  s/crypto_pk_env_t/crypto_pk_t/g;
  s/crypto_dh_env_t/crypto_dh_t/g;
  s/crypto_cipher_env_t/crypto_cipher_t/g;
  s/crypto_digest_env_t/crypto_digest_t/g;

  s/aes_free_cipher/aes_cipher_free/g;
  s/crypto_free_cipher_env/crypto_cipher_free/g;
  s/crypto_free_digest_env/crypto_digest_free/g;
  s/crypto_free_pk_env/crypto_pk_free/g;

  s/_crypto_dh_env_get_dh/_crypto_dh_get_dh/g;
  s/_crypto_new_pk_env_rsa/_crypto_new_pk_from_rsa/g;
  s/_crypto_pk_env_get_evp_pkey/_crypto_pk_get_evp_pkey/g;
  s/_crypto_pk_env_get_rsa/_crypto_pk_get_rsa/g;

  s/crypto_new_cipher_env/crypto_cipher_new/g;
  s/crypto_new_digest_env/crypto_digest_new/g;
  s/crypto_new_digest256_env/crypto_digest256_new/g;
  s/crypto_new_pk_env/crypto_pk_new/g;

  s/crypto_create_crypto_env/crypto_cipher_new/g;

  s/connection_create_listener/connection_listener_new/g;
  s/smartlist_create/smartlist_new/g;
  s/transport_create/transport_new/g;

We're using it incorrectly in many cases, and it doesn't help as far as
we know.

We'll use this to detect changes in CSV options.

This commit is _exactly_ the result of

perl -i -pe 's/\bmemcmp\(/tor_memcmp\(/g' src/*/*.[ch]
perl -i -pe 's/\!\s*tor_memcmp\(/tor_memeq\(/g' src/*/*.[ch]
perl -i -pe 's/0\s*==\s*tor_memcmp\(/tor_memeq\(/g' src/*/*.[ch]
perl -i -pe 's/0\s*!=\s*tor_memcmp\(/tor_memneq\(/g' src/*/*.[ch]
git checkout src/common/di_ops.[ch]
git checkout src/or/test.c
git checkout src/common/test.h

About 860 doxygen-less things remain in 0.2.2

This changes the pqueue API by requiring an additional int in every
structure that we store in a pqueue to hold the index of that structure
within the heap.

The subversion $Id$ fields made every commit force a rebuild of
whatever file got committed.  They were not actually useful for
telling the version of Tor files in the wild.

svn:r17867

Use a consistent naming standard for header file guard macros, taking care not to collide with any system headers.  This tripped us up on Android.

svn:r17805

svn:r17393

identity fingerprints, so it's easier to paste them in.
Suggested by Lucky Green.


svn:r17021

Replace the dummy-use var in SMARTLIST_FOREACH_END() with one that is less likely to confuse analysis tools into thinking we do use after free.  Arguably, (void)x should count as use in suppressing unused variable warnings, but not in generating hey-you-used-a-variable warnings.  Arguably, though, it shouldn't.

svn:r16815

svn:r16802

svn:r16597

 Allow alternate form of SMARTLIST_FOREACH with paired BEGIN and END macros.  This lets the compiler tell us which line an error has occurred on.


svn:r16256

 When we remove old routers, use Bloom filters rather than a digestmap-based set in order to tell which ones we absolutely need to keep.  This will save us roughly a kazillion little short-lived allocations for hash table entries.


svn:r14318

 Add a new SMARTLIST_FOREACH_JOIN macro to iterate through two sorted lists in lockstep.  This happens at least 3 times in the code so far, and is likely to happen more in the future.  Previous attempts to do so proved touchy, tricky, and error-prone: now, we only need to get it right in one place.


svn:r14309

 Better macro documentation


svn:r14298

svn:r13663

 Do the last part of arma's fix for bug 437: Track the origin of every addrmap, and use this info so we can remove all the trackhostexits-originated mappings for a given exit.


svn:r13660

svn:r13649

 Correct handling of nested MAP_FOREACH loops.  There are none of those ATM, but isn't it nice that now they'll be correct?


svn:r13637

svn:r13621

 Simplify rounding logic in bitarray; fix a bug in bitarray_expand().


svn:r13619

 Chris Palmer notes that almost nobody is using smartlist_set_capacity().  Chris Palmer is right.  Remove this basically pointless function.


svn:r13596