Function only used within the hugepage ifdef for Windows so move it there so we
avoid a unused function warning on our Windows CI:

src/ext/equix/hashx/src/virtual_memory.c:30:13: error: 'set_privilege' defined but not used [-Werror=unused-function]
   30 | static bool set_privilege(const char* pszPrivilege, BOOL bEnable) {
      |             ^~~~~~~~~~~~~

Signed-off-by: David Goulet <dgoulet@torproject.org>

Equi-X supports optionally allocating its solver memory using huge
pages, to reduce the virtual memory subsystem overhead required to make
the entire solver buffer live.

Tor doesn't use this feature, since it seems to have no noticeable
performance benefit at this time, but we still included code for it at
compile time. To improve portability, this patch disables huge page
support by default and enables it only in the cmake build system used
for equix benchmarks.

With this patch equix-bench still supports huge pages. Verified using
strace that we're making the hugepage allocation.

There's no fallback for huge pages, so Equi-X initialization will fail
if they are requested and we don't support them for any runtime or
compile-time reason.

Addresses #40843 (NetBSD) but also prevents future porting issues
related to huge pages.

NetBSD includes the idea of a 'maximum protection' per-region,
and an mprotect which exceeds the max protection will be denied.

If we explicitly ask for a maximum which includes execute permission, we
can successfully swap our code buffer's permissions between read-write
and read-execute when each hash program is compiled.

With this patch, the crypto/hashx tests pass on NetBSD 9.
This addresses bug #40844

This path in hashx_vm_alloc_huge() for OpenBSD and NetBSD always
fails without checking its parameter. Fix the warning.

As suggested by @wiz on #40843, let's add an explicit check to
hashx_vm_alloc_huge() that avoids using a Linux-style default
on NetBSD targets.

This doesn't change the questionable Linux-style default,
but a future patch will disable this code by default so it's not a
portability liability.

(This code is in hashx's VM layer but it's actually only relevant to
equix.)

This addresses bug #40843. Another patch will disable huge pages
by default entirely, but this patch is sufficient to fix the NetBSD
build.

Clippy found a transmute that could have been a reborrow.

This is a fix for a very rare buffer overflow in hashx, specific to the
dynamic compiler on aarch64 platforms.

In practice this issue is extremely unlikely to hit randomly, and it's
only been seen in unit tests that supply unusual mock PRNG output to the
program generator. My best attempt at estimating the probability of
hitting the overflow randomly is about 10^-23. Crafting an input with
the intent to overflow can be done only as fast as an exhaustive search,
so long as Blake2B is unbroken.

The root cause is that hashx writes assembly code without any length
checks, and it uses an estimated size rather than an absolute maximum
size to allocate the buffer for compiled code. Some instructions are
much longer than others, especially on aarch64.

The length of the overflow is nearly 300 bytes in the worst synthetic
test cases I've developed so far. Overflow occurs during hashx_make(),
and the subsequent hashx_exec() will always SIGSEGV as the written code
crosses outside the region that's been marked executable. In typical use,
hashx_exec() is called immediately after hashx_make().

This fix increases the buffer size from 1 page to 2 pages on aarch64,
adds an analysis of the compiled code size, and adds runtime checks so we
can gracefully fail on overflow. It also adds a unit test (written in
Rust) that includes a PRNG sequence exercising the overflow. Without
this patch the unit test shows a SIGSEGV on aarch64, with this patch it
runs successfully and matches interpreter output.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>

Fix a couple cases where size_t values were being confused with int.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>

This patch has no effect on the C tor build.

Adds a function hashx_rng_callback() to the hashx API, defined only
when HASHX_RNG_CALLBACK is defined. This is then used in the Rust
wrapper to implement a similar rng_callback().

Included some minimal test cases. This code is intented for
use in cross-compatibility fuzzing tests which drive multiple
implementations of hashx with the same custom Rng stream.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>

The idea behind this is that we may want to start exporting more pieces
of c-tor as Rust crates so that Arti can perform cross compatibility and
comparison testing using Rust tooling.

This turns the 'tor' repo into a Cargo workspace, and adds one crate to
start with: "tor-c-equix", rooted in src/ext/equix. This actually
includes both Equi-X itself and HashX, since there's less overall
duplication if we package these together instead of packaging HashX
separately.

This patch adds a basic safe Rust interface, but doesn't expose any
additional internals for testing purposes.

No changes to the C code here or the normal Tor build system.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>

This addresses issue #40800 and a couple other problems I noticed while
trying to reproduce that one.

The original issue is just a missing cast to void* on the args of
__builtin___clear_cache(), and clang is picky about the implicit cast
between what it considers to be char of different signedness. Original
report is from MacOS but it's also reproducible on other clang targets.

The cmake-based original build system for equix and hashx was a handy
way to run tests, but it suffered from some warnings due to incorrect
application of include_directories().

And lastly, there were some return codes from hashx_exec() that get
ignored on equix when asserts are disabled. It bugged me too much to
just silence this with a (void) cast, since even though this is in the
realm of low-likelyhood programming errors and not true runtime errors, I
don't want to make it easy for the hashx_exec() wrappers to return
values that are dangerously wrong if an error is ignored. I made sure
that even if asserts are disabled, we return values that will cause the
solver and verifier to both fail to validate a potential solution.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>

This change adapts Equi-X to the corresponding HashX API changes that
added HASHX_TRY_COMPILE. The new regularized HashX return codes are
reflected by revised corresponding Equi-X return codes.

Both solve and verify operations now return an error/success code, and a
new equix_solutions_buffer struct includes both the solution buffer
and information about the solution count and hashx implementation.

With this change, it's possible to discern between hash construction
failures (invalid seed) and some external error like an mprotect()
failure.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>

This is an API breaking change to hashx, which modifies the error
handling strategy. The main goal here is to allow unproblematic
recovery from hashx_compile failures.

hashx_alloc can no longer fail for reasons other than memory
allocation. All platform-specific compile failures are now reported via
hashx_make(), in order to both allow later failure and avoid requiring
users of the API to maintain and test multiple failure paths.

Note that late failures may be more common in actual use than early
failures. Early failures represent architectures other than x86_64 and
aarch64. Late failures could represent a number of system configurations
where syscalls are restricted.

The definition of a hashx context no longer tries to overlay storage for
the different types of program, and instead allows one context to always
contain an interpretable description of the program as well as an optional
buffer for compiled code.

The hashx_type enum is now used to mean either a specific type of hash
function or a type of hashx context. You can allocate a context for use
only with interpreted or compiled functions, or you can use
HASHX_TRY_COMPILE to prefer the compiler with an automatic fallback on
the interpreter. After calling hashx_make(), the new hashx_query_type()
can be used if needed to determine which implementation was actually
chosen.

The error return types have been overhauled so that everyone uses the
hashx_result enum, and seed failures vs compile failures are always
clearly distinguishable.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>

This is a minimal portion of the fix for tor issue #40794, in which
hashx segfaults due to denial of mprotect() syscalls at runtime.

Prior to this fix, hashx makes the assumption that if the JIT is
supported on the current architecture, it will also be usable at
runtime. This isn't true if mprotect fails on linux, which it may for
various reasons: the tor built-in sandbox, the shadow simulator, or
external security software that implements a syscall filter.

The necessary error propagation was missing internally in hashx,
causing us to obliviously call into code which was never made
executable. With this fix, hashx_make() will instead fail by returning
zero.

A proper fix will require API changes so that callers can discern
between different types of failures. Zero already means that a program
couldn't be constructed, which requires a different response: choosing a
different seed, vs switching implementations. Callers would also benefit
from a way to use one context (with its already-built program) to
run in either compiled or interpreted mode.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>

The code style in equix and hashx sometimes uses bitwise operators
in place of logical ones in cases where it doesn't really matter
either way. This sometimes annoys our static analyzer tools.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>

This addresses one of the warnings in issue #40792. As far as I can tell
this is a false positive, since the use of "ctx->type" in hashx_free()
can only be hit after the unioned code/program pointer is non-NULL. It's
no big deal to zero this value explicitly to silence the warning though.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>

Fixes some type nitpicks that show up in Tor development builds,
which usually run with -Wall -Werror. Tested on x86_64 and aarch64
for clean build and passing equix-tests + hashx-tests.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>

This replaces the sketchy cmake invocation we had inside configure

The libs are always built and always used in unit tests, but only
included in libtor and tor when --enable-gpl is set.

Signed-off-by: Micah Elizabeth Scott <beth@torproject.org>

Signed-off-by: David Goulet <dgoulet@torproject.org>