Skip to content
GitLab
Projects
Groups
Snippets
/
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
David Goulet
Tor
Commits
a5df9402
Commit
a5df9402
authored
Mar 21, 2019
by
George Kadianakis
Committed by
Nick Mathewson
Mar 22, 2019
Browse files
prob-distr: Decrease false positive rate of stochastic tests.
parent
3871fb73
Changes
2
Hide whitespace changes
Inline
Side-by-side
changes/bug29693
0 → 100644
View file @
a5df9402
o Minor bugfixes (unit tests):
- Decrease the false positive rate of stochastic probability distribution
tests. Fixes bug 29693; bugfix on 0.4.0.1-alpha.
\ No newline at end of file
src/test/test_prob_distr.c
View file @
a5df9402
...
...
@@ -878,11 +878,14 @@ test_uniform_interval(void *arg)
* is higher: 1 - Binom(0; n, alpha) = 1 - (1 - alpha)^n. For n = 10,
* this is about 10%, and for n = 100 it is well over 50%.
*
* We can drive it down by running each test twice, and accepting it if
* it passes at least once; in that case, it is as if we used Binom(2;
* 2, alpha) = alpha^2 as the false positive rate for each test, and
* for n = 10 tests, it would be 0.1%, and for n = 100 tests, still
* only 1%.
* Given that these tests will run with every CI job, we want to drive down the
* false positive rate. We can drive it down by running each test four times,
* and accepting it if it passes at least once; in that case, it is as if we
* used Binom(4; 2, alpha) = alpha^4 as the false positive rate for each test,
* and for n = 10 tests, it would be 9.99999959506e-08. If each CI build has 14
* jobs, then the chance of a CI build failing is 1.39999903326e-06, which
* means that a CI build will break with probability 50% after about 495106
* builds.
*
* The critical value for a chi^2 distribution with 100 degrees of
* freedom and false positive rate alpha = 1% was taken from:
...
...
@@ -895,7 +898,7 @@ test_uniform_interval(void *arg)
static
const
size_t
NSAMPLES
=
100000
;
/* Number of chances we give to the test to succeed. */
static
const
unsigned
NTRIALS
=
2
;
static
const
unsigned
NTRIALS
=
4
;
/* Number of times we want the test to pass per NTRIALS. */
static
const
unsigned
NPASSES_MIN
=
1
;
...
...
Write
Preview
Supports
Markdown
0%
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment