GitLab

While testing an update to the (proposed) TBB port for OpenBSD both I and my partner in torbsd.crime were able to get the instance of tor started by TBB to dump core, but not reliably.

We're using tor 0.2.8.7 under OpenBSD-current (Sept 5 snapshot). I've built myself a package for amd64 from the OpenBSD port with debugging symbols, so I can see what's going on. Under -current you do:

$ cd /usr/ports/net/tor
$ env DEBUG="-ggdb -O0" INSTALL_STRIP= make repackage

and install the resulting /usr/ports/packages/amd64/all/tor-0.2.8.7.tgz package.

Other than that I made no changes to tor itself. The core dump happened both with the standard package (no debug syms) and my package with debug syms.

We die in nodelist.c:836 at the call to the SL_ADD_NEW_IPV6_AP() macro because node->rs appears to be an invalid pointer (node->ri is fine):

(gdb) where
#0  0x000013438bc334a2 in tor_addr_family (a=0x1345c7c3ff58) at address.h:155
#1  0x000013438bc3501c in tor_addr_is_null (addr=0x1345c7c3ff58)
    at src/common/address.c:871
#2  0x000013438bc3526e in tor_addr_is_valid (addr=0x1345c7c3ff58, 
    for_listening=0) at src/common/address.c:932
#3  0x000013438bb1575f in node_get_all_orports (node=0x1345c21f6000)
    at src/or/nodelist.c:836
#4  0x000013438bc29a20 in node_is_a_configured_bridge (node=0x1345c21f6000)
    at src/or/entrynodes.c:1871
#5  0x000013438bc2b74a in any_bridge_supports_microdescriptors ()
    at src/or/entrynodes.c:2486
#6  0x000013438bb0d2ef in we_use_microdescriptors_for_circuits (
    options=0x134681d2f7a0) at src/or/microdesc.c:924
#7  0x000013438bb0d3e9 in usable_consensus_flavor () at src/or/microdesc.c:961
#8  0x000013438bb102e8 in networkstatus_consensus_is_bootstrapping (
    now=1473280922) at src/or/networkstatus.c:1249
#9  0x000013438bc019b2 in find_dl_schedule (dls=0x13438c0185d0, 
    options=0x134681d2f7a0) at src/or/directory.c:3732
#10 0x000013438bc020d0 in download_status_reset (dls=0x13438c0185d0)
    at src/or/directory.c:3950
#11 0x000013438bb114bc in networkstatus_set_current_consensus (
    consensus=0x13468873f000 "network-status-version 3 microdesc\nvote-status consensus\nconsensus-method 20\nvalid-after 2016-09-07 20:00:00\nfresh-until 2016-09-07 21:00:00\nvalid-until 2016-09-07 23:00:00\nvoting-delay 300 300\nclient"..., flavor=0x1345e6fb8470 "microdesc", flags=0) at src/or/networkstatus.c:1679
#12 0x000013438bbfba02 in connection_dir_client_reached_eof (
    conn=0x1346506c2500) at src/or/directory.c:2009
#13 0x000013438bbfda9a in connection_dir_reached_eof (conn=0x1346506c2500)
    at src/or/directory.c:2471
#14 0x000013438bbd32e9 in connection_reached_eof (conn=0x1346506c2500)
    at src/or/connection.c:4841
#15 0x000013438bbd058d in connection_handle_read_impl (conn=0x1346506c2500)
    at src/or/connection.c:3526
#16 0x000013438bbd05d9 in connection_handle_read (conn=0x1346506c2500)
    at src/or/connection.c:3541
#17 0x000013438bb031ec in conn_read_callback (fd=-1, event=2, 
    _conn=0x1346506c2500) at src/or/main.c:803
#18 0x0000134603284cbe in event_base_loop ()
   from /usr/local/lib/libevent_core.so.1.1
#19 0x000013438bb06397 in run_main_loop_once () at src/or/main.c:2543
#20 0x000013438bb064da in run_main_loop_until_done () at src/or/main.c:2589
#21 0x000013438bb062b7 in do_main_loop () at src/or/main.c:2515
#22 0x000013438bb0a0e5 in tor_main (argc=16, argv=0x7f7ffffc01b8)
    at src/or/main.c:3646
#23 0x000013438bb01f3f in main (argc=16, argv=0x7f7ffffc01b8)
    at src/or/tor_main.c:30
(gdb) up
#1  0x000013438bc3501c in tor_addr_is_null (addr=0x1345c7c3ff58)
    at src/common/address.c:871
871	  switch (tor_addr_family(addr)) {
(gdb) up
#2  0x000013438bc3526e in tor_addr_is_valid (addr=0x1345c7c3ff58, 
    for_listening=0) at src/common/address.c:932
932	  return !tor_addr_is_null(addr);
(gdb) up
#3  0x000013438bb1575f in node_get_all_orports (node=0x1345c21f6000)
    at src/or/nodelist.c:836
836	    SL_ADD_NEW_IPV6_AP(node->rs, ipv6_orport, sl, valid);
(gdb) print node->rs
$16 = (routerstatus_t *) 0x1345c7c3ff00
(gdb) print *node->rs
Cannot access memory at address 0x1345c7c3ff00
(gdb) print node->ri
$18 = (routerinfo_t *) 0x134596a7aa00
(gdb) print *node->ri
$19 = {cache_info = {signed_descriptor_body = 0x0, annotations_len = 73, 
    signed_descriptor_len = 2223, 
    signed_descriptor_digest = "§À[º`?ø/\023\005ò\223»Q\004\223j\204íÌ", 
    identity_digest = "\232h¸Z\0021\217N~\207ò\202\2009ûÕ×[\001B", 
    published_on = 1473266407, 
    extra_info_digest = "¡ce8ÃÆ]ü\204^mà *º\220\021\205¹ä", 
    extra_info_digest256 = "¥m\n\231\234\003\230ý\021|ã\035hÊ\025b2 0ÐÐk/\217à\233ò\235\005ÇÇî", signing_key_cert = 0x1346133eb100, ei_dl_status = {
      next_attempt_at = 1473280814, n_download_failures = 0 '\0', 
      n_download_attempts = 0 '\0', schedule = DL_SCHED_GENERIC, 
      want_authority = DL_WANT_ANY_DIRSERVER, 
      increment_on = DL_SCHED_INCREMENT_FAILURE}, 
    saved_location = SAVED_IN_CACHE, saved_offset = 0, routerlist_index = 0, 
    last_listed_as_valid_until = 0, do_not_cache = 0, is_extrainfo = 0, 
    extrainfo_is_bogus = 0, send_unencrypted = 0}, 
  nickname = 0x13459bfe5820 "NYCBUG0", addr = 1114571284, or_port = 9001, 
  dir_port = 9030, ipv6_addr = {family = 0 '\0', addr = {dummy_ = 0, 
      in_addr = {s_addr = 0}, in6_addr = {__u6_addr = {
          __u6_addr8 = '\0' <repeats 15 times>, __u6_addr16 = {0, 0, 0, 0, 0, 
            0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}}}, ipv6_orport = 0, 
  onion_pkey = 0x13465a3d8d20, identity_pkey = 0x134674ecf280, 
  onion_curve25519_pkey = 0x134643b73920, cert_expiration_time = 1473872400, 
  platform = 0x134643b739a0 "Tor 0.2.9.2-alpha on FreeBSD", 
  bandwidthrate = 10240000, bandwidthburst = 15360000, 
  bandwidthcapacity = 7341056, exit_policy = 0x134674ecfd40, 
  ipv6_exit_policy = 0x0, uptime = 3, declared_family = 0x134674ecffb0, 
  contact_info = 0x134643b79780 "Admin <mirror-admin AT nycbug DOT org>", 
  is_hibernating = 0, caches_extra_info = 0, allow_single_hop_exits = 0, 
  wants_to_be_hs_dir = 1, policy_is_reject_star = 1, 
  needs_retest_if_added = 0, supports_tunnelled_dir_requests = 1, 
  omit_from_vote = 0, purpose = 2 '\002'}
(gdb) print node
$20 = (const node_t *) 0x1345c21f6000
(gdb) print *node
$21 = {ht_ent = {hte_next = 0x0, hte_hash = 1201906925}, nodelist_idx = 0,
  identity = "\232hZ\0021\217N~\207202\2009[\001B", md = 0x13463eac4500,
  ri = 0x134596a7aa00, rs = 0x1345c7c3ff00, is_running = 1, is_valid = 1,
  is_fast = 1, is_stable = 1, is_possible_guard = 1, is_exit = 0,
  is_bad_exit = 0, is_hs_dir = 0, name_lookup_warned = 0, rejects_all = 0,
  using_as_guard = 0, ipv6_preferred = 0, country = 5, last_reachable = 0,
  last_reachable6 = 0}

I wish I had more details to offer so far that's all I have.

I've changed my malloc.conf(5) settings since the crash to see if any of the new debug features in OpenBSD's malloc(3) implementation will catch anything (maybe use after free?):

attila@rotfl:~ 18:$ ls -l /etc/malloc.conf
lrwxr-xr-x  1 root  wheel  5 Sep  7 16:55 /etc/malloc.conf -> CFGJU

I've restarted and am hoping to cause this to occur again. Will update this ticket if I learn anything else. Bug me on IRC if you want (I'm attila on #tor-dev).