Using mmap saves about 10 MB of anonymous memory. Using a binary format also
cuts the uncompressed disk space of GeoIP databases in half (~5 MB) too;
compressed size shrinks by 5-15% (since GeoIP databases are highly
compressible, there is more benefit at lower compression levels). A B-tree
would be more efficient but are too complicated to implement in C and aren't
justified by the infrequent geoip lookup.

Officially restricting countries to 2-alpha allows arrays to be statically
instead of dynamically allocated, saving more RAM and reducing code complexity,
in particular by removing routerset_refresh_countries and many smartlists.
Additionally, IPFire has removed A1 country code, so we don't need to store or
handle it anymore. Also, stop storing ?? country in GeoIP files, since Tor
smushes "no data for this IP" and "no country for this IP" together anyways.

A previous version of this change emitted binary GeoIP files directly from
geoip-db-tool. While simpler, that approach does not work well with git; the
default diff is useless, making it hard to see what is changing in the GeoIP
database; additionally, git can potentially compress text deltas more
efficiently than binary deltas. Awk was selected for convert_geoip because it
is already a requirement for configure, and unlike C, does not interfere with
cross-compilation.

Used for geoip-mmap tests, but could be useful elsewhere.

It is unnecessary for the public interface, only routerset_contains_node passes
a legitimate country. Fixes TestingDirAuth* incorrectly ignoring country code
options.

We cap our number of CPU worker threads to at least 2 even if we have a
single core. But also, before we used to always add one extra thread
regardless of the number of core.

This meant that we were off when re-using the get_num_cpus() function
when calculating our onionskin work overhead because we were always off
by one.

This commit makes it that we always use the number of thread our actual
thread pool was configured with.

Fixes #40719

Signed-off-by: David Goulet <dgoulet@torproject.org>

Signed-off-by: David Goulet <dgoulet@torproject.org>

Signed-off-by: David Goulet <dgoulet@torproject.org>

Part of #40712

Signed-off-by: David Goulet <dgoulet@torproject.org>

Created and Rejected connections are ever going up counters. While
Opened connections are gauges going up and down.

Fixes #40712

Signed-off-by: David Goulet <dgoulet@torproject.org>

Signed-off-by: David Goulet <dgoulet@torproject.org>

Rasmus Dahlberg authored Oct 12, 2022 and David Goulet committed Nov 09, 2022

This change mitigates DNS-based website oracles by making the time that
a domain name is cached uncertain (+- 4 minutes of what's measurable).

Resolves TROVE-2021-009.

Fixes #40674

This is part of the fast path so we need to cache consensus parameters
instead of querying it everytime we need to learn a value.

Part of #40704

Signed-off-by: David Goulet <dgoulet@torproject.org>

Until now, there was this magic number (64) used as the maximum number
of tasks a CPU worker can take at once.

This commit makes it a consensus parameter so our future selves can
think of a better value depending on network conditions.

Part of #40704

Signed-off-by: David Goulet <dgoulet@torproject.org>

Transform the hardcoded value ONIONQUEUE_WAIT_CUTOFF into a consensus
parameter so we can control it network wide.

Closes #40704

Signed-off-by: David Goulet <dgoulet@torproject.org>

Part of #40704

Signed-off-by: David Goulet <dgoulet@torproject.org>

Mike Perry authored Nov 08, 2022 and David Goulet committed Nov 09, 2022

Part of #40708.

Mike Perry authored Nov 08, 2022 and David Goulet committed Nov 09, 2022

Part of #40708.

Sam James authored Nov 08, 2022 and David Goulet committed Nov 09, 2022

Clang 16 warns on -Wstrict-prototypes in preparation for C23 which can
among other things, lead to some configure tests silently failing/returning the wrong result.

Fixes this error:
```
-ignoreme: warning: a function declaration without a prototype is deprecated in all versions of C [-Wstrict-prototypes]
+ignoreme: error: a function declaration without a prototype is deprecated in all versions of C [-Werror,-Wstrict-prototypes]
 main ()
```

For more information, see LWN.net [0] or LLVM's Discourse [1], gentoo-dev@ [2],
or the (new) c-std-porting mailing list [3].

[0] https://lwn.net/Articles/913505/
[1] https://discourse.llvm.org/t/configure-script-breakage-with-the-new-werror-implicit-function-declaration/65213
[2] https://archives.gentoo.org/gentoo-dev/message/dd9f2d3082b8b6f8dfbccb0639e6e240
[3] hosted at lists.linux.dev.

Bug: https://bugs.gentoo.org/879747


Signed-off-by: Sam James <sam@gentoo.org>
---

Signed-off-by: David Goulet <dgoulet@torproject.org>

Part of #40708.