Every file in the cache is labeled.  The labels are held in memory;
the bodies are mapped on demand.

These add a tiny bit of structure on top of the regular storagedir
abstractions in order to store key-value lists at the head of each
document.

I'm doing this to storagedir to used config_line_t.

David Goulet authored 7 years ago and Nick Mathewson committed 7 years ago

In order to avoid src/or/hs_service.o to contain no symbols and thus making
clang throw a warning, the functions are now exposed not just to unit tests.

Signed-off-by: David Goulet <dgoulet@torproject.org>

Capture the warning for the unhandled circuit purpose test case, both
to clean up the test log and to confirm that it gets logged.

Taylor Yu authored 8 years ago and Nick Mathewson committed 7 years ago

Check that route_len_for_purpose() (helper for new_route_len())
correctly fails a non-fatal bug assertion if it encounters an
unhandled circuit purpose when it is called with exit node info.

Taylor Yu authored 8 years ago and Nick Mathewson committed 7 years ago

Add a new helper function route_len_for_purpose(), which explicitly
lists all of the known circuit purposes for a circuit with a chosen
exit node (unlike previously, where the default route length for a
chosen exit was DEFAULT_ROUTE_LEN + 1 except for two purposes).  Add a
non-fatal assertion for unhandled purposes that conservatively returns
DEFAULT_ROUTE_LEN + 1.

Add copious comments documenting which circuits need an extra hop and
why.

Thanks to nickm and dgoulet for providing background information.

The old implementation had duplicated code in a bunch of places, and
it interspersed spool-management with resource management.  The new
implementation should make it easier to add new resource types and
maintain the spooling code.

Closing ticket 21651.

George Kadianakis authored 8 years ago and Nick Mathewson committed 8 years ago

When calculating max sampled size, Tor would only count the number of
bridges in torrc, without considering that our state file might already
have sampled bridges in it. This caused problems when people swap
bridges, since the following error would trigger:

         [warn] Not expanding the guard sample any further; just hit the
                maximum sample threshold of 1

This prevents an i386 compilation warning and fixes bug 21828. Bug not
in any released Tor.

Fixes bug21799.

Leak caused by clean_up_backtrace_handler not being called
on shutdown.

Alexander Hansen Færøy authored 8 years ago and Nick Mathewson committed 8 years ago

This patch changes the way we decide when to check for whether it's time
to rotate and/or expiry our onion keys. Due to proposal #274 we can now
have the keys rotate at different frequencies than before and we thus
do the check once an hour when our Tor daemon is running in server mode.

This should allow us to quickly notice if the network consensus
parameter have changed while we are running instead of having to wait
until the current parameters timeout value have passed.

See: See: https://bugs.torproject.org/21641

Alexander Hansen Færøy authored 8 years ago and Nick Mathewson committed 8 years ago

This patch adds a new timer that is executed when it is time to expire
our current set of old onion keys. Because of proposal #274 this can no
longer be assumed to be at the same time we rotate our onion keys since
they will be updated less frequently.

See: https://bugs.torproject.org/21641

Alexander Hansen Færøy authored 8 years ago and Nick Mathewson committed 8 years ago

This patch adds an API to get the current grace period, in days, defined
as the consensus parameter "onion-key-grace-period-days".

As per proposal #274 the values for "onion-key-grace-period-days" is a
default value of 7 days, a minimum value of 1 day, and a maximum value
defined by other consensus parameter "onion-key-rotation-days" also
defined in days.

See: https://bugs.torproject.org/21641

Alexander Hansen Færøy authored 8 years ago and Nick Mathewson committed 8 years ago

This patch turns `MIN_ONION_KEY_LIFETIME` into a new function
`get_onion_key_lifetime()` which gets its value from a network consensus
parameter named "onion-key-rotation-days". This allows us to tune the
value at a later point in time with no code modifications.

We also bump the default onion key lifetime from 7 to 28 days as per
proposal #274.

See: https://bugs.torproject.org/21641

Alexander Hansen Færøy authored 8 years ago and Alexander Hansen Færøy committed 8 years ago

This patch fixes a regression described in bug #21757 that first
appeared after commit 6e78ede7 which was an attempt to fix bug #21654.

When switching from buffered I/O to direct file descriptor I/O our
output strings from get_string_from_pipe() might contain newline
characters (\n). In this patch we modify tor_get_lines_from_handle() to
ensure that the function splits the newly read string at the newline
character and thus might return multiple lines from a single call to
get_string_from_pipe().

Additionally, we add a test case to test_util_string_from_pipe() to
ensure that get_string_from_pipe() correctly returns multiple lines in a
single call.

See: https://bugs.torproject.org/21757
See: https://bugs.torproject.org/21654

It was very error-prone to maintain this by hand.

We could use one of these for holding "junk" descriptors and
unparseable things -- but we'll _need_ it for having cached
consensuses and diffs between them.

Thanks, jenkins!

There was a frequent block of code that did "find the next router
line, see if we've hit the end of the list, get the ID hash from the
line, and enforce well-ordering."  Per Ahf's review, I'm extracting
it to its own function.

Previously, we operated on smartlists of NUL-terminated strings,
which required us to copy both inputs to produce the NUL-terminated
strings.  Then we copied parts of _those_ inputs to produce an
output smartlist of NUL-terminated strings.  And finally, we
concatenated everything into a final resulting string.

This implementation, instead, uses a pointer-and-extent pattern to
represent each line as a pointer into the original inputs and a
length.  These line objects are then added by reference into the
output. No actual bytes are copied from the original strings until
we finally concatenate the final result together.

Bookkeeping structures and newly allocated strings (like ed
commands) are allocated inside a memarea, to avoid needless mallocs
or complicated should-I-free-this-or-not bookkeeping.

In my measurements, this improves CPU performance by something like
18%.  The memory savings should be much, much higher.