Trac issueshttps://gitlab.torproject.org/legacy/trac/-/issues2020-06-13T15:52:26Zhttps://gitlab.torproject.org/legacy/trac/-/issues/33673Use the right DLLs and pkg-config path on Appveyor2020-06-13T15:52:26ZteorUse the right DLLs and pkg-config path on AppveyorWe want to future-proof our Appveyor CI against dll and pkg-config issues.
Split off from #33643, which is the urgent CI fix.We want to future-proof our Appveyor CI against dll and pkg-config issues.
Split off from #33643, which is the urgent CI fix.Tor: 0.3.5.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/33643Appveyor: OpenSSL version mismatch in unit tests, 2020 edition2020-06-13T15:52:21ZteorAppveyor: OpenSSL version mismatch in unit tests, 2020 editionIt's happened again:
```
OpenSSL library version 1.1.1d did not begin with header version 1.1.1e.
```
https://ci.appveyor.com/project/torproject/tor/builds/31549942/job/v0i9svtg78gqln1v#L6380
Just like #32449, #28574, #28399 and #25942....It's happened again:
```
OpenSSL library version 1.1.1d did not begin with header version 1.1.1e.
```
https://ci.appveyor.com/project/torproject/tor/builds/31549942/job/v0i9svtg78gqln1v#L6380
Just like #32449, #28574, #28399 and #25942.
We think our tests are fragile, because they are not copying the necessary libraries into `${env:build}/src/test` from `C:/mingw32/lib`:
```
ssl
crypto
lzma
event
zstd
```
We already copy zlib and ssp at https://github.com/ahf/tor/blob/master/.appveyor.yml#L98-L99 .
These libraries might have different dll prefixes or suffixes, for example, OpenSSL is:
```
/mingw32/bin/libcrypto-1_1.dll
/mingw32/bin/libssl-1_1.dll
```
https://packages.msys2.org/package/mingw-w64-i686-openssl
We might also want to copy these libraries into the same directory as the tor executable `${env:build}/src/app`, before we run the tests that launch tor.Tor: 0.3.5.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/31397Appveyor: pacman.exe : error: msys: signature … is invalid2020-06-13T15:44:15ZteorAppveyor: pacman.exe : error: msys: signature … is invalidThe msys signature is invalid, which causes all our Appveyor builds to fail.
This issue first occurred around 0645 UTC Monday 12 August 2019.
It appears to be an external issue.
```
downloading msys.db.sig...
pacman.exe : error: msys: s...The msys signature is invalid, which causes all our Appveyor builds to fail.
This issue first occurred around 0645 UTC Monday 12 August 2019.
It appears to be an external issue.
```
downloading msys.db.sig...
pacman.exe : error: msys: signature from "Alexey Pavlov (Alexpux) <alexpux@gmail.com>" is invalid
At line:2 char:5
+ & $commandPath $args 2>&1
+ ~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (error: msys: si...om>" is invalid:String) [], RemoteException
+ FullyQualifiedErrorId : NativeCommandError
error:
failed to update msys (invalid or corrupted database (PGP signature))
error: failed to synchronize all databases
Command executed with exception: failed to update msys (invalid or corrupted database (PGP signature))
error: failed to synchronize all databases
```Tor: 0.4.1.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/29602Speed up builds using Appveyor's build cache2020-06-13T15:38:39ZteorSpeed up builds using Appveyor's build cacheWe can use up to 1 GB (compressed) build cache in Appveyor across all our jobs:
https://www.appveyor.com/docs/build-cache/
Is there something we could cache to speed up builds?
It would have to be less than 200 MB compressed.We can use up to 1 GB (compressed) build cache in Appveyor across all our jobs:
https://www.appveyor.com/docs/build-cache/
Is there something we could cache to speed up builds?
It would have to be less than 200 MB compressed.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/28659Appveyor CI: error removing temporary directories - permission denied2020-06-13T15:34:54Zrl1987Appveyor CI: error removing temporary directories - permission deniedhttps://ci.appveyor.com/project/rl1987/tor/builds/20640384/job/3lk0nnv48m6vc5ni
```
consdiffmgr/cleanup_old_diffs: [forking] Error removing C:\Users\appveyor\AppData\Local\Temp\1\tor_test_1340_oylvawto\datadir_cdm_uj2f3yo3\diff-cache\10...https://ci.appveyor.com/project/rl1987/tor/builds/20640384/job/3lk0nnv48m6vc5ni
```
consdiffmgr/cleanup_old_diffs: [forking] Error removing C:\Users\appveyor\AppData\Local\Temp\1\tor_test_1340_oylvawto\datadir_cdm_uj2f3yo3\diff-cache\1000: Permission denied
Error removing C:\Users\appveyor\AppData\Local\Temp\1\tor_test_1340_oylvawto\datadir_cdm_uj2f3yo3\diff-cache\1003: Permission denied
Error removing C:\Users\appveyor\AppData\Local\Temp\1\tor_test_1340_oylvawto\datadir_cdm_uj2f3yo3\diff-cache\1006: Permission denied
Error removing C:\Users\appveyor\AppData\Local\Temp\1\tor_test_1340_oylvawto\datadir_cdm_uj2f3yo3\diff-cache\1013: Permission denied
Error removing directory C:\Users\appveyor\AppData\Local\Temp\1\tor_test_1340_oylvawto\datadir_cdm_uj2f3yo3\diff-cache: Directory not empty
Error removing directory C:\Users\appveyor\AppData\Local\Temp\1\tor_test_1340_oylvawto\datadir_cdm_uj2f3yo3: Directory not empty
Error removing directory C:\Users\appveyor\AppData\Local\Temp\1\tor_test_1340_oylvawto: Directory not empty
OK
consdiffmgr/validate: [forking] OK
```Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/28574Appveyor: OpenSSL unit test fails with header and library version mismatch2020-06-13T15:52:21ZteorAppveyor: OpenSSL unit test fails with header and library version mismatchI'm guessing that 1.1.1a and 1.1.1 are compatible, though?
```
crypto/openssl_version: [forking]
FAIL ../src/test/test_crypto.c:238: OpenSSL library version 1.1.1 did not begin with header version 1.1.1a.
[openssl_version FAILED]
``...I'm guessing that 1.1.1a and 1.1.1 are compatible, though?
```
crypto/openssl_version: [forking]
FAIL ../src/test/test_crypto.c:238: OpenSSL library version 1.1.1 did not begin with header version 1.1.1a.
[openssl_version FAILED]
```
https://ci.appveyor.com/project/torproject/tor/build/1.0.1625/job/gibgc64fp4hxsf2h?fullLog=true#L3064Tor: 0.3.4.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/28459Appveyor: always make the logs into build artifacts2020-06-13T15:34:13ZteorAppveyor: always make the logs into build artifactsThey're only about 50kB, and sometimes we want to have logs for a successful build.They're only about 50kB, and sometimes we want to have logs for a successful build.Tor: 0.3.4.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/28457Appveyor: make doesn't stop when the first child process fails2020-06-13T15:34:13ZteorAppveyor: make doesn't stop when the first child process failsThis is probably a Windows child process handling bug in make.
We might be able to work around it by swapping to the MSYS2 or mingw make.
Discovered in #28399.This is probably a Windows child process handling bug in make.
We might be able to work around it by swapping to the MSYS2 or mingw make.
Discovered in #28399.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/28454Actually use zstd on Appveyor2020-06-13T15:34:12ZteorActually use zstd on AppveyorWe install zstd, but configure can't find it.We install zstd, but configure can't find it.Tor: 0.4.0.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/28399Build is broken on Appveyor CI2020-06-13T15:52:21Zrl1987Build is broken on Appveyor CIhttps://ci.appveyor.com/project/torproject/tor/builds/20213306
```
bash.exe : ../src/lib/tls/tortls_openssl.c: In function 'tor_tls_context_new':
At line:2 char:5
+ & $commandPath $args 2>&1
+ ~~~~~~~~~~~~~~~~~~~~~~~~~
+ Cat...https://ci.appveyor.com/project/torproject/tor/builds/20213306
```
bash.exe : ../src/lib/tls/tortls_openssl.c: In function 'tor_tls_context_new':
At line:2 char:5
+ & $commandPath $args 2>&1
+ ~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (../src/lib/tls/...s_context_new'::String) [], RemoteException
+ FullyQualifiedErrorId : NativeCommandError
../src/lib/tls/tortls_openssl.c:545:35: error: implicit declaration of function 'TLS_method'; did you mean 'DTLS_method'? [-Werror=implicit-function-declaration]
if (!(result->ctx = SSL_CTX_new(TLS_method())))
^~~~~~~~~~
DTLS_method
../src/lib/tls/tortls_openssl.c:545:35: error: nested extern declaration of 'TLS_method' [-Werror=nested-externs]
../src/lib/tls/tortls_openssl.c:545:35: error: passing argument 1 of 'SSL_CTX_new' makes pointer from integer without a cast [-Werror=int-conversion]
In file included from ../src/lib/tls/tortls_openssl.c:48:0:
C:/OpenSSL-Win32/include/openssl/ssl.h:2131:10: note: expected 'const SSL_METHOD * {aka const struct ssl_method_st *}' but argument is of type 'int'
SSL_CTX *SSL_CTX_new(const SSL_METHOD *meth);
^~~~~~~~~~~
../src/lib/tls/tortls_openssl.c:554:3: error: implicit declaration of function 'SSL_CTX_set_security_level'; did you mean 'SSL_CTX_set_verify_depth'? [-Werror=implicit-function-declaration]
SSL_CTX_set_security_level(result->ctx, 1);
^~~~~~~~~~~~~~~~~~~~~~~~~~
SSL_CTX_set_verify_depth
../src/lib/tls/tortls_openssl.c:554:3: error: nested extern declaration of 'SSL_CTX_set_security_level' [-Werror=nested-externs]
../src/lib/tls/tortls_openssl.c: In function 'prune_v2_cipher_list':
../src/lib/tls/tortls_openssl.c:810:25: error: initialization makes pointer from integer without a cast [-Werror=int-conversion]
const SSL_METHOD *m = TLS_method();
^~~~~~~~~~
../src/lib/tls/tortls_openssl.c: In function 'tor_tls_client_is_using_v2_ciphers':
../src/lib/tls/tortls_openssl.c:919:13: error: implicit declaration of function 'SSL_get_client_ciphers'; did you mean 'SSL_get_current_cipher'? [-Werror=implicit-function-declaration]
ciphers = SSL_get_client_ciphers(ssl);
^~~~~~~~~~~~~~~~~~~~~~
SSL_get_current_cipher
../src/lib/tls/tortls_openssl.c:919:13: error: nested extern declaration of 'SSL_get_client_ciphers' [-Werror=nested-externs]
../src/lib/tls/tortls_openssl.c:919:11: error: assignment makes pointer from integer without a cast [-Werror=int-conversion]
ciphers = SSL_get_client_ciphers(ssl);
^
../src/lib/tls/tortls_openssl.c: In function 'tor_tls_get_tlssecrets':
../src/lib/tls/tortls_openssl.c:1623:36: error: implicit declaration of function 'SSL_get_server_random'; did you mean 'SSL_get_servername'? [-Werror=implicit-function-declaration]
const size_t server_random_len = SSL_get_server_random(ssl, NULL, 0);
^~~~~~~~~~~~~~~~~~~~~
SSL_get_servername
../src/lib/tls/tortls_openssl.c:1623:36: error: nested extern declaration of 'SSL_get_server_random' [-Werror=nested-externs]
../src/lib/tls/tortls_openssl.c:1624:36: error: implicit declaration of function 'SSL_get_client_random'; did you mean 'SSL_get_client_CA_list'? [-Werror=implicit-function-declaration]
const size_t client_random_len = SSL_get_client_random(ssl, NULL, 0);
^~~~~~~~~~~~~~~~~~~~~
SSL_get_client_CA_list
../src/lib/tls/tortls_openssl.c:1624:36: error: nested extern declaration of 'SSL_get_client_random' [-Werror=nested-externs]
../src/lib/tls/tortls_openssl.c:1625:33: error: implicit declaration of function 'SSL_SESSION_get_master_key'; did you mean 'SSL_SESSION_get_time'? [-Werror=implicit-function-declaration]
const size_t master_key_len = SSL_SESSION_get_master_key(session, NULL, 0);
^~~~~~~~~~~~~~~~~~~~~~~~~~
SSL_SESSION_get_time
../src/lib/tls/tortls_openssl.c:1625:33: error: nested extern declaration of 'SSL_SESSION_get_master_key' [-Werror=nested-externs]
```Tor: 0.3.5.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/28318Appveyor: build on Windows Server 2012 R2 and Windows Server 20162020-06-13T15:33:46ZteorAppveyor: build on Windows Server 2012 R2 and Windows Server 2016Appveyor provides build images with Windows Server 2012 R2 (the default) and Windows Server 2016. These images are called:
* Visual Studio 2015
* Visual Studio 2017
https://www.appveyor.com/docs/windows-images-software/#operating-system...Appveyor provides build images with Windows Server 2012 R2 (the default) and Windows Server 2016. These images are called:
* Visual Studio 2015
* Visual Studio 2017
https://www.appveyor.com/docs/windows-images-software/#operating-system
Our builds work on these images:
https://ci.appveyor.com/project/teor2345/tor/builds/20046114
So we should probably build on both of them.Tor: 0.3.5.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/27943Broken Appveyor build2020-06-13T15:32:25Zrl1987Broken Appveyor build```
checking for file conflicts...
error:
failed to commit transaction (conflicting files)
```
https://ci.appveyor.com/project/torproject/tor/build/1.0.1020/job/o62dw3iklb8da8wv```
checking for file conflicts...
error:
failed to commit transaction (conflicting files)
```
https://ci.appveyor.com/project/torproject/tor/build/1.0.1020/job/o62dw3iklb8da8wvTor: 0.3.5.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/26979Appveyor CI IRC shows the wrong branch for pull requests2020-06-13T15:28:52ZteorAppveyor CI IRC shows the wrong branch for pull requestsFor example, the branch that Neel opened a pull request for is b23588, based on master:
```
11:49 appveyor-ci: torproject/tor master 1752b17 - Neel Chauhan: Add changes file for Bug #23588
11:49 appveyor-ci: Build #1.0.450 failed. Detail...For example, the branch that Neel opened a pull request for is b23588, based on master:
```
11:49 appveyor-ci: torproject/tor master 1752b17 - Neel Chauhan: Add changes file for Bug #23588
11:49 appveyor-ci: Build #1.0.450 failed. Details: https://ci.appveyor.com/project/torproject/tor/build/1.0.450
```
If I can't fix this in a few minutes tomorrow, I'll leave it for later.Tor: 0.3.4.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/26535Appveyor fails test_ntor.sh and test_hs_ntor.sh2020-06-13T15:27:18ZGeorge KadianakisAppveyor fails test_ntor.sh and test_hs_ntor.sh```
FAIL: src/test/test_ntor.sh
2571===========================
2572
2573Traceback (most recent call last):
2574 File "C:/projects/tor/i686-w64-mingw32/../src/test/ntor_ref.py", line 402, in <module>
2575 test_tor()
2576 File "C:/pr...```
FAIL: src/test/test_ntor.sh
2571===========================
2572
2573Traceback (most recent call last):
2574 File "C:/projects/tor/i686-w64-mingw32/../src/test/ntor_ref.py", line 402, in <module>
2575 test_tor()
2576 File "C:/projects/tor/i686-w64-mingw32/../src/test/ntor_ref.py", line 368, in test_tor
2577 c2s_msg, c_state = tor_client1(node_id, pubkey_B)
2578 File "C:/projects/tor/i686-w64-mingw32/../src/test/ntor_ref.py", line 347, in tor_client1
2579 stdout=subprocess.PIPE)
2580 File "C:\msys64\mingw32\lib\python3.6\subprocess.py", line 709, in __init__
2581 restore_signals, start_new_session)
2582 File "C:\msys64\mingw32\lib\python3.6\subprocess.py", line 971, in _execute_child
2583 args = list2cmdline(args)
2584 File "C:\msys64\mingw32\lib\python3.6\subprocess.py", line 461, in list2cmdline
2585 needquote = (" " in arg) or ("\t" in arg) or not arg
2586TypeError: a bytes-like object is required, not 'str'
2587OK
2588FAIL src/test/test_ntor.sh (exit status: 1)
2589
2590
FAIL: src/test/test_hs_ntor.sh
2591==============================
2592
2593DONE: python dance [b'Cc\x0e"k\'*\xc4\x8a\x18\xc2\xfcN\xac\x8e(\xaa\x14\xb1\xccqqCN"\x9f\x9b\xd5W@\x94O']
2594Traceback (most recent call last):
2595 File "C:/projects/tor/i686-w64-mingw32/../src/test/hs_ntor_ref.py", line 424, in <module>
2596 do_little_t_tor_ntor_test()
2597 File "C:/projects/tor/i686-w64-mingw32/../src/test/hs_ntor_ref.py", line 310, in do_little_t_tor_ntor_test
2598 client_ephemeral_enc_privkey, subcredential)
2599 File "C:/projects/tor/i686-w64-mingw32/../src/test/hs_ntor_ref.py", line 248, in tor_client1
2600 stdout=subprocess.PIPE)
2601 File "C:\msys64\mingw32\lib\python3.6\subprocess.py", line 709, in __init__
2602 restore_signals, start_new_session)
2603 File "C:\msys64\mingw32\lib\python3.6\subprocess.py", line 971, in _execute_child
2604 args = list2cmdline(args)
2605 File "C:\msys64\mingw32\lib\python3.6\subprocess.py", line 461, in list2cmdline
2606 needquote = (" " in arg) or ("\t" in arg) or not arg
2607TypeError: a bytes-like object is required, not 'str'
2608FAIL src/test/test_hs_ntor.sh (exit status: 1)
2609
2610
```Tor: 0.3.4.x-finalNick MathewsonNick Mathewson