Commit 35bbf2e4 authored by teor (Tim Wilson-Brown)'s avatar teor (Tim Wilson-Brown)
Browse files

Prop210: Add schedules for simultaneous client consensus downloads

Prop210: Add attempt-based connection schedules

Existing tor schedules increment the schedule position on failure,
then retry the connection after the scheduled time.

To make multiple simultaneous connections, we need to increment the
schedule position when making each attempt, then retry a (potentially
simultaneous) connection after the scheduled time.

(Also change find_dl_schedule_and_len to find_dl_schedule, as it no
longer takes or returns len.)

Prop210: Add multiple simultaneous consensus downloads for clients

Make connections on TestingClientBootstrapConsensus*DownloadSchedule,
incrementing the schedule each time the client attempts to connect.

Check if the number of downloads is less than
TestingClientBootstrapConsensusMaxInProgressTries before trying any
more connections.
parent d3546aa9
o Major features (consensus downloads):
- Schedule multiple in-progress consensus downloads during client
bootstrap. Use the first one that starts downloading, close the
rest. This reduces failures when authorities are slow or down.
With #15775, it reduces failures due to fallback churn.
Implements #4483 (reduce failures when authorities are down).
Patch by "teor".
Implements IPv4 portions of proposal #210 by "mikeperry" and
"teor".
......@@ -2281,10 +2281,18 @@ The following options are used for running a testing Tor network.
TestingClientDownloadSchedule 0, 0, 5, 10, 15, 20, 30, 60
TestingServerConsensusDownloadSchedule 0, 0, 5, 10, 15, 20, 30, 60
TestingClientConsensusDownloadSchedule 0, 0, 5, 10, 15, 20, 30, 60
TestingClientBootstrapConsensusAuthorityDownloadSchedule 0, 2,
4 (for 40 seconds), 8, 16, 32, 60
TestingClientBootstrapConsensusFallbackDownloadSchedule 0, 1,
4 (for 40 seconds), 8, 16, 32, 60
TestingClientBootstrapConsensusAuthorityOnlyDownloadSchedule 0, 1,
4 (for 40 seconds), 8, 16, 32, 60
TestingBridgeDownloadSchedule 60, 30, 30, 60
TestingClientMaxIntervalWithoutRequest 5 seconds
TestingDirConnectionMaxStall 30 seconds
TestingConsensusMaxDownloadTries 80
TestingClientBootstrapConsensusMaxDownloadTries 80
TestingClientBootstrapConsensusAuthorityOnlyMaxDownloadTries 80
TestingDescriptorMaxDownloadTries 80
TestingMicrodescMaxDownloadTries 80
TestingCertMaxDownloadTries 80
......@@ -2345,6 +2353,36 @@ The following options are used for running a testing Tor network.
requires that **TestingTorNetwork** is set. (Default: 0, 0, 60, 300, 600,
1800, 3600, 3600, 3600, 10800, 21600, 43200)
[[TestingClientBootstrapConsensusAuthorityDownloadSchedule]] **TestingClientBootstrapConsensusAuthorityDownloadSchedule** __N__,__N__,__...__::
Schedule for when clients should download consensuses from authorities if
they are bootstrapping (that is, they don't have a usable, reasonably live
consensus). Only used by clients fetching from a list of fallback
directory mirrors. This schedule is advanced by (potentially concurrent)
connection attempts, unlike other schedules, which are advanced by
connection failures. Changing this schedule requires that
**TestingTorNetwork** is set. (Default: 10, 11, 3600, 10800, 25200, 54000,
111600, 262800)
[[TestingClientBootstrapConsensusFallbackDownloadSchedule]] **TestingClientBootstrapConsensusFallbackDownloadSchedule** __N__,__N__,__...__::
Schedule for when clients should download consensuses from fallback
directory mirrors if they are bootstrapping (that is, they don't have a
usable, reasonably live consensus). Only used by clients fetching from a
list of fallback directory mirrors. This schedule is advanced by
(potentially concurrent) connection attempts, unlike other schedules, which
are advanced by connection failures. Changing this schedule requires that
**TestingTorNetwork** is set. (Default: 0, 1, 4, 11, 3600, 10800, 25200,
54000, 111600, 262800)
[[TestingClientBootstrapConsensusAuthorityOnlyDownloadSchedule]] **TestingClientBootstrapConsensusAuthorityOnlyDownloadSchedule** __N__,__N__,__...__::
Schedule for when clients should download consensuses from authorities if
they are bootstrapping (that is, they don't have a usable, reasonably live
consensus). Only used by clients which don't have or won't fetch from a
list of fallback directory mirrors. This schedule is advanced by
(potentially concurrent) connection attempts, unlike other schedules,
which are advanced by connection failures. Changing this schedule requires
that **TestingTorNetwork** is set. (Default: 0, 3, 7, 3600, 10800, 25200,
54000, 111600, 262800)
[[TestingBridgeDownloadSchedule]] **TestingBridgeDownloadSchedule** __N__,__N__,__...__::
Schedule for when clients should download bridge descriptors. Changing this
requires that **TestingTorNetwork** is set. (Default: 3600, 900, 900, 3600)
......@@ -2361,9 +2399,24 @@ The following options are used for running a testing Tor network.
5 minutes)
[[TestingConsensusMaxDownloadTries]] **TestingConsensusMaxDownloadTries** __NUM__::
Try this often to download a consensus before giving up. Changing
Try this many times to download a consensus before giving up. Changing
this requires that **TestingTorNetwork** is set. (Default: 8)
[[TestingClientBootstrapConsensusMaxDownloadTries]] **TestingClientBootstrapConsensusMaxDownloadTries** __NUM__::
Try this many times to download a consensus while bootstrapping using
fallback directory mirrors before giving up. Changing this requires that
**TestingTorNetwork** is set. (Default: 7)
[[TestingClientBootstrapConsensusMaxDownloadTries]] **TestingClientBootstrapConsensusMaxDownloadTries** __NUM__::
Try this many times to download a consensus while bootstrapping using
only authorities before giving up. Changing this requires that
**TestingTorNetwork** is set. (Default: 4)
[[TestingClientBootstrapConsensusMaxInProgressTries]] **TestingClientBootstrapConsensusMaxInProgressTries** __NUM__::
Try this many simultaneous connections to download a consensus before
waiting for one to complete, timeout, or error out. Changing this
requires that **TestingTorNetwork** is set. (Default: 4)
[[TestingDescriptorMaxDownloadTries]] **TestingDescriptorMaxDownloadTries** __NUM__::
Try this often to download a server descriptor before giving up.
Changing this requires that **TestingTorNetwork** is set. (Default: 8)
......
......@@ -336,6 +336,32 @@ typedef uint32_t uintptr_t;
#endif /* time_t_is_signed */
#endif /* ifndef(TIME_MAX) */
#ifndef TIME_MIN
#ifdef TIME_T_IS_SIGNED
#if (SIZEOF_TIME_T == SIZEOF_INT)
#define TIME_MIN ((time_t)INT_MIN)
#elif (SIZEOF_TIME_T == SIZEOF_LONG)
#define TIME_MIN ((time_t)LONG_MIN)
#elif (SIZEOF_TIME_T == 8)
#define TIME_MIN ((time_t)INT64_MIN)
#else
#error "Can't define (signed) TIME_MIN"
#endif
#else
/* Unsigned case */
#if (SIZEOF_TIME_T == 4)
#define TIME_MIN ((time_t)UINT32_MIN)
#elif (SIZEOF_TIME_T == 8)
#define TIME_MIN ((time_t)UINT64_MIN)
#else
#error "Can't define (unsigned) TIME_MIN"
#endif
#endif /* time_t_is_signed */
#endif /* ifndef(TIME_MIN) */
#ifndef SIZE_MAX
#if (SIZEOF_SIZE_T == 4)
#define SIZE_MAX UINT32_MAX
......
......@@ -475,10 +475,40 @@ static config_var_t option_vars_[] = {
V(TestingClientConsensusDownloadSchedule, CSV_INTERVAL, "0, 0, 60, "
"300, 600, 1800, 3600, 3600, 3600, "
"10800, 21600, 43200"),
/* With the TestingClientBootstrapConsensus*Download* below:
* Clients with only authorities will try:
* - 3 authorities over 10 seconds, then wait 60 minutes.
* Clients with authorities and fallbacks will try:
* - 2 authorities and 4 fallbacks over 21 seconds, then wait 60 minutes.
* Clients will also retry when an application request arrives.
* After a number of failed reqests, clients retry every 3 days + 1 hour.
*
* Clients used to try 2 authorities over 10 seconds, then wait for
* 60 minutes or an application request.
*
* When clients have authorities and fallbacks available, they use these
* schedules: (we stagger the times to avoid thundering herds) */
V(TestingClientBootstrapConsensusAuthorityDownloadSchedule, CSV_INTERVAL,
"10, 11, 3600, 10800, 25200, 54000, 111600, 262800" /* 3 days + 1 hour */),
V(TestingClientBootstrapConsensusFallbackDownloadSchedule, CSV_INTERVAL,
"0, 1, 4, 11, 3600, 10800, 25200, 54000, 111600, 262800"),
/* When clients only have authorities available, they use this schedule: */
V(TestingClientBootstrapConsensusAuthorityOnlyDownloadSchedule, CSV_INTERVAL,
"0, 3, 7, 3600, 10800, 25200, 54000, 111600, 262800"),
/* We don't want to overwhelm slow networks (or mirrors whose replies are
* blocked), but we also don't want to fail if only some mirrors are
* blackholed. Clients will try 3 directories simultaneously.
* (Relays never use simultaneous connections.) */
V(TestingClientBootstrapConsensusMaxInProgressTries, UINT, "3"),
V(TestingBridgeDownloadSchedule, CSV_INTERVAL, "3600, 900, 900, 3600"),
V(TestingClientMaxIntervalWithoutRequest, INTERVAL, "10 minutes"),
V(TestingDirConnectionMaxStall, INTERVAL, "5 minutes"),
V(TestingConsensusMaxDownloadTries, UINT, "8"),
/* Since we try connections rapidly and simultaneously, we can afford
* to give up earlier. (This protects against overloading directories.) */
V(TestingClientBootstrapConsensusMaxDownloadTries, UINT, "7"),
/* We want to give up much earlier if we're only using authorities. */
V(TestingClientBootstrapConsensusAuthorityOnlyMaxDownloadTries, UINT, "4"),
V(TestingDescriptorMaxDownloadTries, UINT, "8"),
V(TestingMicrodescMaxDownloadTries, UINT, "8"),
V(TestingCertMaxDownloadTries, UINT, "8"),
......@@ -525,10 +555,18 @@ static const config_var_t testing_tor_network_defaults[] = {
"15, 20, 30, 60"),
V(TestingClientConsensusDownloadSchedule, CSV_INTERVAL, "0, 0, 5, 10, "
"15, 20, 30, 60"),
V(TestingClientBootstrapConsensusAuthorityDownloadSchedule, CSV_INTERVAL,
"0, 2, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 8, 16, 32, 60"),
V(TestingClientBootstrapConsensusFallbackDownloadSchedule, CSV_INTERVAL,
"0, 1, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 8, 16, 32, 60"),
V(TestingClientBootstrapConsensusAuthorityOnlyDownloadSchedule, CSV_INTERVAL,
"0, 1, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 8, 16, 32, 60"),
V(TestingBridgeDownloadSchedule, CSV_INTERVAL, "60, 30, 30, 60"),
V(TestingClientMaxIntervalWithoutRequest, INTERVAL, "5 seconds"),
V(TestingDirConnectionMaxStall, INTERVAL, "30 seconds"),
V(TestingConsensusMaxDownloadTries, UINT, "80"),
V(TestingClientBootstrapConsensusMaxDownloadTries, UINT, "80"),
V(TestingClientBootstrapConsensusAuthorityOnlyMaxDownloadTries, UINT, "80"),
V(TestingDescriptorMaxDownloadTries, UINT, "80"),
V(TestingMicrodescMaxDownloadTries, UINT, "80"),
V(TestingCertMaxDownloadTries, UINT, "80"),
......@@ -3749,10 +3787,16 @@ options_validate(or_options_t *old_options, or_options_t *options,
CHECK_DEFAULT(TestingClientDownloadSchedule);
CHECK_DEFAULT(TestingServerConsensusDownloadSchedule);
CHECK_DEFAULT(TestingClientConsensusDownloadSchedule);
CHECK_DEFAULT(TestingClientBootstrapConsensusAuthorityDownloadSchedule);
CHECK_DEFAULT(TestingClientBootstrapConsensusFallbackDownloadSchedule);
CHECK_DEFAULT(TestingClientBootstrapConsensusAuthorityOnlyDownloadSchedule);
CHECK_DEFAULT(TestingBridgeDownloadSchedule);
CHECK_DEFAULT(TestingClientMaxIntervalWithoutRequest);
CHECK_DEFAULT(TestingDirConnectionMaxStall);
CHECK_DEFAULT(TestingConsensusMaxDownloadTries);
CHECK_DEFAULT(TestingClientBootstrapConsensusMaxDownloadTries);
CHECK_DEFAULT(TestingClientBootstrapConsensusAuthorityOnlyMaxDownloadTries);
CHECK_DEFAULT(TestingClientBootstrapConsensusMaxInProgressTries);
CHECK_DEFAULT(TestingDescriptorMaxDownloadTries);
CHECK_DEFAULT(TestingMicrodescMaxDownloadTries);
CHECK_DEFAULT(TestingCertMaxDownloadTries);
......@@ -3827,11 +3871,41 @@ options_validate(or_options_t *old_options, or_options_t *options,
}
if (options->TestingConsensusMaxDownloadTries < 2) {
REJECT("TestingConsensusMaxDownloadTries must be greater than 1.");
REJECT("TestingConsensusMaxDownloadTries must be greater than 2.");
} else if (options->TestingConsensusMaxDownloadTries > 800) {
COMPLAIN("TestingConsensusMaxDownloadTries is insanely high.");
}
if (options->TestingClientBootstrapConsensusMaxDownloadTries < 2) {
REJECT("TestingClientBootstrapConsensusMaxDownloadTries must be greater "
"than 2."
);
} else if (options->TestingClientBootstrapConsensusMaxDownloadTries > 800) {
COMPLAIN("TestingClientBootstrapConsensusMaxDownloadTries is insanely "
"high.");
}
if (options->TestingClientBootstrapConsensusAuthorityOnlyMaxDownloadTries
< 2) {
REJECT("TestingClientBootstrapConsensusAuthorityOnlyMaxDownloadTries must "
"be greater than 2."
);
} else if (
options->TestingClientBootstrapConsensusAuthorityOnlyMaxDownloadTries
> 800) {
COMPLAIN("TestingClientBootstrapConsensusAuthorityOnlyMaxDownloadTries is "
"insanely high.");
}
if (options->TestingClientBootstrapConsensusMaxInProgressTries < 1) {
REJECT("TestingClientBootstrapConsensusMaxInProgressTries must be greater "
"than 0.");
} else if (options->TestingClientBootstrapConsensusMaxInProgressTries
> 100) {
COMPLAIN("TestingClientBootstrapConsensusMaxInProgressTries is insanely "
"high.");
}
if (options->TestingDescriptorMaxDownloadTries < 2) {
REJECT("TestingDescriptorMaxDownloadTries must be greater than 1.");
} else if (options->TestingDescriptorMaxDownloadTries > 800) {
......
......@@ -3443,26 +3443,54 @@ connection_dir_finished_connecting(dir_connection_t *conn)
}
/** Decide which download schedule we want to use based on descriptor type
* in <b>dls</b> and whether we are acting as directory <b>server</b>, and
* then return a list of int pointers defining download delays in seconds.
* Helper function for download_status_increment_failure() and
* download_status_reset(). */
* in <b>dls</b> and <b>options</b>.
* Then return a list of int pointers defining download delays in seconds.
* Helper function for download_status_increment_failure(),
* download_status_reset(), and download_status_increment_attempt(). */
static const smartlist_t *
find_dl_schedule_and_len(download_status_t *dls, int server)
{
find_dl_schedule(download_status_t *dls, const or_options_t *options)
{
/* XX/teor Replace with dir_server_mode from #12538 */
const int dir_server = options->DirPort_set;
const int multi_d = networkstatus_consensus_can_use_multiple_directories(
options);
const int we_are_bootstrapping = networkstatus_consensus_is_boostrapping(
time(NULL));
const int use_fallbacks = networkstatus_consensus_can_use_extra_fallbacks(
options);
switch (dls->schedule) {
case DL_SCHED_GENERIC:
if (server)
return get_options()->TestingServerDownloadSchedule;
else
return get_options()->TestingClientDownloadSchedule;
if (dir_server) {
return options->TestingServerDownloadSchedule;
} else {
return options->TestingClientDownloadSchedule;
}
case DL_SCHED_CONSENSUS:
if (server)
return get_options()->TestingServerConsensusDownloadSchedule;
else
return get_options()->TestingClientConsensusDownloadSchedule;
if (!multi_d) {
return options->TestingServerConsensusDownloadSchedule;
} else {
if (we_are_bootstrapping) {
if (!use_fallbacks) {
/* A bootstrapping client without extra fallback directories */
return
options->TestingClientBootstrapConsensusAuthorityOnlyDownloadSchedule;
} else if (dls->want_authority) {
/* A bootstrapping client with extra fallback directories, but
* connecting to an authority */
return
options->TestingClientBootstrapConsensusAuthorityDownloadSchedule;
} else {
/* A bootstrapping client connecting to extra fallback directories
*/
return
options->TestingClientBootstrapConsensusFallbackDownloadSchedule;
}
} else {
return options->TestingClientConsensusDownloadSchedule;
}
}
case DL_SCHED_BRIDGE:
return get_options()->TestingBridgeDownloadSchedule;
return options->TestingBridgeDownloadSchedule;
default:
tor_assert(0);
}
......@@ -3471,54 +3499,168 @@ find_dl_schedule_and_len(download_status_t *dls, int server)
return NULL;
}
/** Called when an attempt to download <b>dls</b> has failed with HTTP status
/* Find the current delay for dls based on schedule.
* Set dls->next_attempt_at based on now, and return the delay.
* Helper for download_status_increment_failure and
* download_status_increment_attempt. */
STATIC int
download_status_schedule_get_delay(download_status_t *dls,
const smartlist_t *schedule,
time_t now)
{
tor_assert(dls);
tor_assert(schedule);
int delay = INT_MAX;
uint8_t dls_schedule_position = (dls->increment_on
== DL_SCHED_INCREMENT_ATTEMPT
? dls->n_download_attempts
: dls->n_download_failures);
if (dls_schedule_position < smartlist_len(schedule))
delay = *(int *)smartlist_get(schedule, dls_schedule_position);
else if (dls_schedule_position == IMPOSSIBLE_TO_DOWNLOAD)
delay = INT_MAX;
else
delay = *(int *)smartlist_get(schedule, smartlist_len(schedule) - 1);
/* A negative delay makes no sense. Knowing that delay is
* non-negative allows us to safely do the wrapping check below. */
tor_assert(delay >= 0);
/* Avoid now+delay overflowing INT_MAX, by comparing with a subtraction
* that won't overflow (since delay is non-negative). */
if (delay < INT_MAX && now <= INT_MAX - delay) {
dls->next_attempt_at = now+delay;
} else {
dls->next_attempt_at = TIME_MAX;
}
return delay;
}
/* Log a debug message about item, which increments on increment_action, has
* incremented dls_n_download_increments times. The message varies based on
* was_schedule_incremented (if not, not_incremented_response is logged), and
* the values of increment, dls_next_attempt_at, and now.
* Helper for download_status_increment_failure and
* download_status_increment_attempt. */
static void
download_status_log_helper(const char *item, int was_schedule_incremented,
const char *increment_action,
const char *not_incremented_response,
uint8_t dls_n_download_increments, int increment,
time_t dls_next_attempt_at, time_t now)
{
if (item) {
if (!was_schedule_incremented)
log_debug(LD_DIR, "%s %s %d time(s); I'll try again %s.",
item, increment_action, (int)dls_n_download_increments,
not_incremented_response);
else if (increment == 0)
log_debug(LD_DIR, "%s %s %d time(s); I'll try again immediately.",
item, increment_action, (int)dls_n_download_increments);
else if (dls_next_attempt_at < TIME_MAX)
log_debug(LD_DIR, "%s %s %d time(s); I'll try again in %d seconds.",
item, increment_action, (int)dls_n_download_increments,
(int)(dls_next_attempt_at-now));
else
log_debug(LD_DIR, "%s %s %d time(s); Giving up for a while.",
item, increment_action, (int)dls_n_download_increments);
}
}
/** Determine when a failed download attempt should be retried.
* Called when an attempt to download <b>dls</b> has failed with HTTP status
* <b>status_code</b>. Increment the failure count (if the code indicates a
* real failure) and set <b>dls</b>-\>next_attempt_at to an appropriate time
* in the future. */
* real failure, or if we're a server) and set <b>dls</b>-\>next_attempt_at to
* an appropriate time in the future and return it.
* If <b>dls->increment_on</b> is DL_SCHED_INCREMENT_ATTEMPT, increment the
* failure count, and return a time in the far future for the next attempt (to
* avoid an immediate retry). */
time_t
download_status_increment_failure(download_status_t *dls, int status_code,
const char *item, int server, time_t now)
{
const smartlist_t *schedule;
int increment;
int increment = -1;
tor_assert(dls);
/* only count the failure if it's permanent, or we're a server */
if (status_code != 503 || server) {
if (dls->n_download_failures < IMPOSSIBLE_TO_DOWNLOAD-1)
++dls->n_download_failures;
}
schedule = find_dl_schedule_and_len(dls, server);
if (dls->increment_on == DL_SCHED_INCREMENT_FAILURE) {
/* We don't find out that a failure-based schedule has attempted a
* connection until that connection fails.
* We'll never find out about successful connections, but this doesn't
* matter, because schedules are reset after a successful download.
*/
if (dls->n_download_attempts < IMPOSSIBLE_TO_DOWNLOAD-1)
++dls->n_download_attempts;
if (dls->n_download_failures < smartlist_len(schedule))
increment = *(int *)smartlist_get(schedule, dls->n_download_failures);
else if (dls->n_download_failures == IMPOSSIBLE_TO_DOWNLOAD)
increment = INT_MAX;
else
increment = *(int *)smartlist_get(schedule, smartlist_len(schedule) - 1);
/* only return a failure retry time if this schedule increments on failures
*/
const smartlist_t *schedule = find_dl_schedule(dls, get_options());
increment = download_status_schedule_get_delay(dls, schedule, now);
}
if (increment < INT_MAX)
dls->next_attempt_at = now+increment;
else
dls->next_attempt_at = TIME_MAX;
download_status_log_helper(item, !dls->increment_on, "failed",
"concurrently", dls->n_download_failures,
increment, dls->next_attempt_at, now);
if (item) {
if (increment == 0)
log_debug(LD_DIR, "%s failed %d time(s); I'll try again immediately.",
item, (int)dls->n_download_failures);
else if (dls->next_attempt_at < TIME_MAX)
log_debug(LD_DIR, "%s failed %d time(s); I'll try again in %d seconds.",
item, (int)dls->n_download_failures,
(int)(dls->next_attempt_at-now));
else
log_debug(LD_DIR, "%s failed %d time(s); Giving up for a while.",
item, (int)dls->n_download_failures);
if (dls->increment_on == DL_SCHED_INCREMENT_ATTEMPT) {
/* stop this schedule retrying on failure, it will launch concurrent
* connections instead */
return TIME_MAX;
} else {
return dls->next_attempt_at;
}
}
/** Determine when the next download attempt should be made when using an
* attempt-based (potentially concurrent) download schedule.
* Called when an attempt to download <b>dls</b> is being initiated.
* Increment the attempt count and set <b>dls</b>-\>next_attempt_at to an
* appropriate time in the future and return it.
* If <b>dls->increment_on</b> is DL_SCHED_INCREMENT_FAILURE, don't increment
* the attempts, and return a time in the far future (to avoid launching a
* concurrent attempt). */
time_t
download_status_increment_attempt(download_status_t *dls, const char *item,
time_t now)
{
int delay = -1;
tor_assert(dls);
if (dls->increment_on == DL_SCHED_INCREMENT_FAILURE) {
/* this schedule should retry on failure, and not launch any concurrent
attempts */
log_info(LD_BUG, "Tried to launch an attempt-based connection on a "
"failure-based schedule.");
return TIME_MAX;
}
if (dls->n_download_attempts < IMPOSSIBLE_TO_DOWNLOAD-1)
++dls->n_download_attempts;
const smartlist_t *schedule = find_dl_schedule(dls, get_options());
delay = download_status_schedule_get_delay(dls, schedule, now);
download_status_log_helper(item, dls->increment_on, "attempted",
"on failure", dls->n_download_attempts,
delay, dls->next_attempt_at, now);
return dls->next_attempt_at;
}
/** Reset <b>dls</b> so that it will be considered downloadable
* immediately, and/or to show that we don't need it anymore.
*
* Must be called to initialise a download schedule, otherwise the zeroth item
* in the schedule will never be used.
*
* (We find the zeroth element of the download schedule, and set
* next_attempt_at to be the appropriate offset from 'now'. In most
* cases this means setting it to 'now', so the item will be immediately
......@@ -3527,14 +3669,16 @@ download_status_increment_failure(download_status_t *dls, int status_code,
void
download_status_reset(download_status_t *dls)
{
if (dls->n_download_failures == IMPOSSIBLE_TO_DOWNLOAD)
if (dls->n_download_failures == IMPOSSIBLE_TO_DOWNLOAD
|| dls->n_download_attempts == IMPOSSIBLE_TO_DOWNLOAD)
return; /* Don't reset this. */
const smartlist_t *schedule = find_dl_schedule_and_len(
dls, get_options()->DirPort_set);
const smartlist_t *schedule = find_dl_schedule(dls, get_options());
dls->n_download_failures = 0;
dls->n_download_attempts = 0;
dls->next_attempt_at = time(NULL) + *(int *)smartlist_get(schedule, 0);
/* Don't reset dls->want_authority or dls->increment_on */
}
/** Return the number of failures on <b>dls</b> since the last success (if
......@@ -3545,6 +3689,22 @@ download_status_get_n_failures(const download_status_t *dls)
return dls->n_download_failures;
}
/** Return the number of attempts to download <b>dls</b> since the last success
* (if any). This can differ from download_status_get_n_failures() due to
* outstanding concurrent attempts. */
int
download_status_get_n_attempts(const download_status_t *dls)
{
return dls->n_download_attempts;
}
/** Return the next time to attempt to download <b>dls</b>. */
time_t
download_status_get_next_attempt_at(const download_status_t *dls)
{
return dls->next_attempt_at;
}
/** Called when one or more routerdesc (or extrainfo, if <b>was_extrainfo</b>)
* fetches have failed (with uppercase fingerprints listed in <b>failed</b>,
* either as descriptor digests or as identity digests based on
......
......@@ -92,6 +92,8 @@ int router_supports_extrainfo(const char *identity_digest, int is_authority);
time_t download_status_increment_failure(download_status_t *dls,
int status_code, const char *item,
int server, time_t now);
time_t download_status_increment_attempt(download_status_t *dls,
const char *item, time_t now);
/** Increment the failure count of the download_status_t <b>dls</b>, with
* the optional status code <b>sc</b>. */
#define download_status_failed(dls, sc) \
......@@ -107,8 +109,9 @@ static INLINE int
download_status_is_ready(download_status_t *dls, time_t now,
int max_failures)
{
return (dls->n_download_failures <= max_failures
&& dls->next_attempt_at <= now);
int under_failure_limit = (dls->n_download_failures <= max_failures
&& dls->n_download_attempts <= max_failures);
return (under_failure_limit && dls->next_attempt_at <= now);
}
static void download_status_mark_impossible(download_status_t *dl);
......@@ -117,9 +120,12 @@ static INLINE void
download_status_mark_impossible(download_status_t *dl)
{
dl->n_download_failures = IMPOSSIBLE_TO_DOWNLOAD;
dl->n_download_attempts = IMPOSSIBLE_TO_DOWNLOAD;
}
int download_status_get_n_failures(const download_status_t *dls);
int download_status_get_n_attempts(const download_status_t *dls);
time_t download_status_get_next_attempt_at(const download_status_t *dls);
#ifdef TOR_UNIT_TESTS
/* Used only by directory.c and test_dir.c */
......@@ -133,6 +139,9 @@ STATIC int directory_handle_command_get(dir_connection_t *conn,
const char *headers,
const char *req_body,
size_t req_body_len);
STATIC int download_status_schedule_get_delay(download_status_t *dls,
const smartlist_t *schedule,
time_t now);
#endif
#endif
......
......@@ -1876,18 +1876,29 @@ check_for_reachability_bw_callback(time_t now, const or_options_t *options)
static int
fetch_networkstatus_callback(time_t now, const or_options_t *options)
{
/* 2c. Every minute (or every second if TestingTorNetwork), check
* whether we want to download any networkstatus documents. */
/* 2c. Every minute (or every second if TestingTorNetwork, or during
* client bootstrap), check whether we want to download any networkstatus
* documents. */
/* How often do we check whether we should download network status
* documents? */
#define networkstatus_dl_check_interval(o) ((o)->TestingTorNetwork ? 1 : 60)
const int we_are_bootstrapping = networkstatus_consensus_is_boostrapping(
now);
const int prefer_mirrors = !directory_fetches_from_authorities(
get_options());
int networkstatus_dl_check_interval = 60;
/* check more often when testing, or when bootstrapping from mirrors
* (connection limits prevent too many connections being made) */
if (options->TestingTorNetwork
|| (we_are_bootstrapping && prefer_mirrors)) {
networkstatus_dl_check_interval = 1;
}
if (should_delay_dir_fetches(options, NULL))