Skip to content

Wrap the exitmap binary main in an execution guard

Arturo Filastò requested to merge art/exitmap:fix/exitmap-main into main

Not doing so leads to exitmap failing on macOS and windows where the default is to use spawn instead of fork for multiprocessing (see: https://docs.python.org/3/library/multiprocessing.html#contexts-and-start-methods).

Specifically I was getting the following backtrace when running any module on macOS:

$ ./bin/exitmap oonicheck
2023-05-04 13:45:26,357 exitmap [INFO] Attempting to invoke Tor process in directory "/tmp/exitmap_tor_datadir-art".  This might take a while.
2023-05-04 13:45:26,357 exitmap [INFO] No first hop given.  Using randomly determined first hops for circuits.
2023-05-04 13:45:26,577 util [INFO] Tor Bootstrapped 0% (starting): Starting
2023-05-04 13:45:27,579 util [INFO] Tor Bootstrapped 5% (conn): Connecting to a relay
2023-05-04 13:45:27,617 util [INFO] Tor Bootstrapped 10% (conn_done): Connected to a relay
2023-05-04 13:45:27,659 util [INFO] Tor Bootstrapped 14% (handshake): Handshaking with a relay
2023-05-04 13:45:27,770 util [INFO] Tor Bootstrapped 15% (handshake_done): Handshake with a relay done
2023-05-04 13:45:27,770 util [INFO] Tor Bootstrapped 20% (onehop_create): Establishing an encrypted direc#!/usr/bin/env python3
tory connection
2023-05-04 13:45:27,809 util [INFO] Tor Bootstrapped 25% (requesting_status): Asking for networkstatus consensus
2023-05-04 13:45:27,848 util [INFO] Tor Bootstrapped 30% (loading_status): Loading networkstatus consensus
2023-05-04 13:45:28,245 util [INFO] Tor Bootstrapped 40% (loading_keys): Loading authority key certs
2023-05-04 13:45:28,410 util [INFO] Tor Bootstrapped 45% (requesting_descriptors): Asking for relay descriptors
2023-05-04 13:45:28,709 util [INFO] Tor Bootstrapped 50% (loading_descriptors): Loading relay descriptors
2023-05-04 13:45:32,794 util [INFO] Tor Bootstrapped 56% (loading_descriptors): Loading relay descriptors
2023-05-04 13:45:33,316 util [INFO] Tor Bootstrapped 61% (loading_descriptors): Loading relay descriptors
2023-05-04 13:45:33,843 util [INFO] Tor Bootstrapped 69% (loading_descriptors): Loading relay descriptors
2023-05-04 13:45:34,058 util [INFO] Tor Bootstrapped 75% (enough_dirinfo): Loaded enough directory info to build circuits
2023-05-04 13:45:34,058 exitmap [INFO] Successfully started Tor process (PID=28437).
2023-05-04 13:45:34,085 exitmap [INFO] Running module 'oonicheck'.
<module 'modules.oonicheck' from '/Users/art/code/Tor/exitmap/src/modules/oonicheck.py'>
2023-05-04 13:45:34,086 exitmap [INFO] Selecting destinations depending on the module.
2023-05-04 13:45:37,232 relayselector [INFO] 1632 out of 1924 exit candidates meet all filter conditions.
2023-05-04 13:45:37,251 exitmap [INFO] Successfully selected exit relays after 0:00:03.165317.
2023-05-04 13:45:37,252 exitmap [INFO] Running actually the module.
2023-05-04 13:45:37,394 exitmap [INFO] Attempting to invoke Tor process in directory "/tmp/exitmap_tor_datadir-art".  This might take a while.
2023-05-04 13:45:37,394 exitmap [INFO] No first hop given.  Using randomly determined first hops for circuits.
2023-05-04 13:45:42,423 exitmap [ERROR] Couldn't launch Tor: Process terminated: Acting on config options left us in a broken state. Dying. (on Tor 0.4.7.13 ).  Maybe try again?
Removing tmp files...
Removing tmp files...
Traceback (most recent call last):
  File "/Users/art/code/Tor/exitmap/./bin/exitmap", line 38, in <module>
    main()
  File "/Users/art/code/Tor/exitmap/src/exitmap.py", line 281, in main
    run_module(module_name, args, controller, socks_port, stats)
  File "/Users/art/code/Tor/exitmap/src/exitmap.py", line 382, in run_module
    handler = EventHandler(controller, module, socks_port, stats,
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/art/code/Tor/exitmap/src/eventhandler.py", line 164, in __init__
    self.manager = multiprocessing.Manager()
                   ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/python@3.11/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/context.py", line 57, in Manager
    m.start()
  File "/usr/local/Cellar/python@3.11/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/managers.py", line 567, in start
    self._address = reader.recv()
                    ^^^^^^^^^^^^^
  File "/usr/local/Cellar/python@3.11/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/connection.py", line 249, in recv
    buf = self._recv_bytes()
          ^^^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/python@3.11/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/connection.py", line 413, in _recv_bytes
    buf = self._recv(4)
          ^^^^^^^^^^^^^
  File "/usr/local/Cellar/python@3.11/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/connection.py", line 382, in _recv
    raise EOFError
EOFError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/art/code/Tor/exitmap/./bin/exitmap", line 46, in <module>
    shutil.rmtree(args.tor_dir)
  File "/usr/local/Cellar/python@3.11/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/shutil.py", line 722, in rmtree
    onerror(os.lstat, path, sys.exc_info())
  File "/usr/local/Cellar/python@3.11/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/shutil.py", line 720, in rmtree
    orig_st = os.lstat(path, dir_fd=dir_fd)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/exitmap_tor_datadir-art'

What can be seen from this stacktrace is that the line "Attempting to invoke Tor process in directory" is printed twice, which means that the bootstrap_tor function is being invoked twice. I suspect this is happening due to the difference in how multiprocessing behaves on macOS/Windows compared to Linux and is attempting to import the ./bin/exitmap binary which is calling the main twice.

Wrapping the main in an import guard solves the issue and I think is anyways a best practice regardless of the platform.

Merge request reports