Commit 9338847b authored by Nick Mathewson's avatar Nick Mathewson 🎨
Browse files

Write a bunch more test for doc/WritingTests

parent ebff18d1
......@@ -32,25 +32,97 @@ set STEM_SOURCE_DIR to the checkout, and run "make test-stem".
To run the Chutney tests as well, fetch chutney from the git repository,
set CHUTNEY_PATH to the checkout, and run "make test-network".
To run all of the above, run "make test-full".
To run all of the above, plus tests that require a working connection to the
internet, run "make test-full-online".
=== Running particular subtests
XXXX WRITEME
The Tor unit tests are divided into separate programs and a couple of
bundled unit test programs.
Separate programs are easy. For example, to run the memwipe tests in
isolation, you just run ./src/test/test-memwipe .
To run tests within the unit test programs, you can specify the name
of the test. The string ".." can be used as a wildcard at the end of the
test name. For example, to run all the cell format tests, enter
"./src/test/test cellfmt/..". To run
Many tests that need to mess with global state run in forked subprocesses in
order to keep from contaminating one another. But when debugging a failing test,
you might want to run it without forking a subprocess. To do so, use the
"--no-fork" option with a single test. (If you specify it along with
multiple tests, they might interfere.)
You can turn on logging in the unit tests by passing one of "--debug",
"--info", "--notice", or "--warn". By default only errors are displayed.
Unit tests are divided into "./src/test/test" and "./src/test/test-slow".
The former are those that should finish in a few seconds; the latter tend to
take more time, and may include CPU-intensive operations, deliberate delays,
and stuff like that.
=== Finding test coverage
When you configure Tor with the --enable-coverage option, it should
build with support for coverage in the unit tests, and in a special
"tor-cov" binary. If you launch
"tor-cov" binary.
Then, run the tests you'd like to see coverage from. If you have old
coverage output, you may need to run "reset-gcov" first.
Now you've got a bunch of files scattered around your build directories
called "*.gcda". In order to extract the coverage output from them, make a
temporary directory for them and run "./scripts/test/coverage ${TMPDIR}",
where ${TMPDIR} is the temporary directory you made. This will create a
".gcov" file for each source file under tests, containing that file's source
annotated with the number of times the tests hit each line. (You'll need to
have gcov installed.)
You can get a summary of the test coverage for each file by running
"./scripts/test/cov-display ${TMPDIR}/*" . Each line lists the file's name,
the number of uncovered lines, the number of uncovered lines, and the
coverage percentage.
For a summary of the test coverage for each _function_, run
"./scripts/test/cov-display -f ${TMPDIR}/*" .
=== Comparing test coverage
Sometimes it's useful to compare test coverage for a branch you're writing to
coverage from another branch (such as git master, for example). But you
can't run "diff" on the two coverage outputs directly, since the actual
number of times each line is executed aren't so important, and aren't wholly
deterministic.
Instead, follow the instructions above for each branch, creating a separate
temporary directory for each. Then, run "./scripts/test/cov-diff ${D1}
${D2}", where D1 and D2 are the directories you want to compare. This will
produce a diff of the two directories, with all lines normalized to be either
covered or uncovered.
To count new or modified uncovered lines in D2, you can run:
"./scripts/test/cov-diff ${D1} ${D2}" | grep '^+ *\#' |wc -l
XXXX "make test-network" doesn't know about "tor-cov"; you don't get
XXXX coverage from that yet, unless you do "cp src/or/tor-cov
XXXX src/or/tor" before you run it.
What kinds of test should I write?
----------------------------------
XXXX writeme.
Integration testing and unit testing are complementary: it's probably a
good idea to make sure that your code is hit by both if you can.
If your code is very-low level, and its behavior is easily described in
terms of a relation between inputs and outputs, or a set of state
transitions, then it's a natural fit for unit tests. (If not, please
consider refactoring it until most of it _is_ a good fit for unit
tests!)
If your code adds new externally visible functionality to Tor, it would
be great to have a test for that functionality. That's where
integration tests more usually come in.
Unit and regression tests: Does this function do what it's supposed to?
-----------------------------------------------------------------------
......@@ -117,7 +189,7 @@ Sometimes you need to test a function, but you don't want to expose
it outside its usual module.
To support this, Tor's build system compiles a testing version of
teach module, with extra identifiers exposed. If you want to
each module, with extra identifiers exposed. If you want to
declare a function as static but available for testing, use the
macro "STATIC" instead of "static". Then, make sure there's a
macro-protected declaration of the function in the module's header.
......@@ -134,13 +206,47 @@ CRYPTO_CURVE25519_PRIVATE, so they can see this declaration.
=== Mock functions for testing in isolation
Often we want to test that a function works right, but the function depends
on other functions whose behavior is hard to observe, or whose
Often we want to test that a function works right, but the function to
be tested depends on other functions whose behavior is hard to observe,
or which require a working Tor network, or something like that.
XXXX WRITEME
To write tests for this case, you can replace the underlying functions
with testing stubs while your unit test is running. You need to declare
the underlying function as 'mockable', as follows:
=== Advanced techniques: Namespaces
MOCK_DECL(returntype, functionname, (argument list));
and then later implement it as:
MOCK_IMPL(returntype, functionname, (argument list))
{
/* implementation here */
}
For example, if you had a 'connect to remote server' function, you could
declare it as:
MOCK_DECL(int, connect_to_remote, (const char *name, status_t *status));
When you declare a function this way, it will be declared as normal in
regular builds, but when the module is built for testing, it is declared
as a function pointer initialized to the actual implementation.
In your tests, if you want to override the function with a temporary
replacement, you say:
MOCK(functionname, replacement_function_name);
And later, you can restore the original function with:
UNMOCK(functionname);
For more information, see the definitions of this mocking logic in
testsupport.h.
=== Advanced techniques: Namespaces
XXXX write this. danah boyd made us some really awesome stuff here.
......
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment