v3 onion service wrongly considers Invalid signature for service descriptor signing key: expired
I host on a server 2 onion services (one v2 and one v3).
Exact Tor version is: Tor version 0.3.5.0-alpha-dev
Suddenly, the v3 onion service went down. This was showing in the log file:
Sep 23 14:04:00.000 [notice] Tor has successfully opened a circuit. Looks like client functionality is working.
Sep 23 14:04:02.000 [warn] Invalid signature for service descriptor signing key: expired
Sep 23 14:04:02.000 [warn] tor_bug_occurred_(): Bug: ../src/feature/hs/hs_descriptor.c:2661: hs_desc_encode_descriptor: Non-fatal assertion !(ret < 0) failed. (on Tor 0.3.5.0-alpha-dev )
Sep 23 14:04:02.000 [warn] Bug: Non-fatal assertion !(ret < 0) failed in hs_desc_encode_descriptor at ../src/feature/hs/hs_descriptor.c:2661. Stack trace: (on Tor 0.3.5.0-alpha-dev )
Sep 23 14:04:02.000 [warn] Bug: /usr/bin/tor(log_backtrace_impl+0x47) [0x55e9e1761257] (on Tor 0.3.5.0-alpha-dev )
Sep 23 14:04:02.000 [warn] Bug: /usr/bin/tor(tor_bug_occurred_+0xc0) [0x55e9e175ca60] (on Tor 0.3.5.0-alpha-dev )
Sep 23 14:04:02.000 [warn] Bug: /usr/bin/tor(hs_desc_encode_descriptor+0x106) [0x55e9e1667996] (on Tor 0.3.5.0-alpha-dev )
Sep 23 14:04:02.000 [warn] Bug: /usr/bin/tor(hs_service_run_scheduled_events+0x1ab9) [0x55e9e166ff49] (on Tor 0.3.5.0-alpha-dev )
Sep 23 14:04:02.000 [warn] Bug: /usr/bin/tor(+0x562e1) [0x55e9e15cd2e1] (on Tor 0.3.5.0-alpha-dev )
Sep 23 14:04:02.000 [warn] Bug: /usr/bin/tor(+0x5dc11) [0x55e9e15d4c11] (on Tor 0.3.5.0-alpha-dev )
Sep 23 14:04:02.000 [warn] Bug: /usr/lib/x86_64-linux-gnu/libevent-2.0.so.5(event_base_loop+0x6a0) [0x7f246c99f5a0] (on Tor 0.3.5.0-alpha-dev )
Sep 23 14:04:02.000 [warn] Bug: /usr/bin/tor(do_main_loop+0x265) [0x55e9e15d11b5] (on Tor 0.3.5.0-alpha-dev )
Sep 23 14:04:02.000 [warn] Bug: /usr/bin/tor(tor_run_main+0x11f5) [0x55e9e15d3a35] (on Tor 0.3.5.0-alpha-dev )
Sep 23 14:04:02.000 [warn] Bug: /usr/bin/tor(tor_main+0x3a) [0x55e9e15cba0a] (on Tor 0.3.5.0-alpha-dev )
Sep 23 14:04:02.000 [warn] Bug: /usr/bin/tor(main+0x19) [0x55e9e15cb589] (on Tor 0.3.5.0-alpha-dev )
Sep 23 14:04:02.000 [warn] Bug: /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf1) [0x7f246b1fb2e1] (on Tor 0.3.5.0-alpha-dev )
Sep 23 14:04:02.000 [warn] Bug: /usr/bin/tor(_start+0x2a) [0x55e9e15cb5da] (on Tor 0.3.5.0-alpha-dev )
Sep 23 14:04:02.000 [warn] tor_bug_occurred_(): Bug: ../src/feature/hs/hs_service.c:2816: upload_descriptor_to_hsdir: Non-fatal assertion !(service_encode_descriptor(service, desc, &desc->signing_kp, &encoded_desc) < 0) failed. (on Tor $
Sep 23 14:04:02.000 [warn] Bug: Non-fatal assertion !(service_encode_descriptor(service, desc, &desc->signing_kp, &encoded_desc) < 0) failed in upload_descriptor_to_hsdir at ../src/feature/hs/hs_service.c:2816. Stack trace: (on Tor 0.3.$
Sep 23 14:04:02.000 [warn] Bug: /usr/bin/tor(log_backtrace_impl+0x47) [0x55e9e1761257] (on Tor 0.3.5.0-alpha-dev )
Sep 23 14:04:02.000 [warn] Bug: /usr/bin/tor(tor_bug_occurred_+0xc0) [0x55e9e175ca60] (on Tor 0.3.5.0-alpha-dev )
Sep 23 14:04:02.000 [warn] Bug: /usr/bin/tor(hs_service_run_scheduled_events+0x1d5b) [0x55e9e16701eb] (on Tor 0.3.5.0-alpha-dev )
Sep 23 14:04:02.000 [warn] Bug: /usr/bin/tor(+0x562e1) [0x55e9e15cd2e1] (on Tor 0.3.5.0-alpha-dev )
Sep 23 14:04:02.000 [warn] Bug: /usr/bin/tor(+0x5dc11) [0x55e9e15d4c11] (on Tor 0.3.5.0-alpha-dev )
Sep 23 14:04:02.000 [warn] Bug: /usr/lib/x86_64-linux-gnu/libevent-2.0.so.5(event_base_loop+0x6a0) [0x7f246c99f5a0] (on Tor 0.3.5.0-alpha-dev )
Sep 23 14:04:02.000 [warn] Bug: /usr/bin/tor(do_main_loop+0x265) [0x55e9e15d11b5] (on Tor 0.3.5.0-alpha-dev )
Sep 23 14:04:02.000 [warn] Bug: /usr/bin/tor(tor_run_main+0x11f5) [0x55e9e15d3a35] (on Tor 0.3.5.0-alpha-dev )
Sep 23 14:04:02.000 [warn] Bug: /usr/bin/tor(tor_main+0x3a) [0x55e9e15cba0a] (on Tor 0.3.5.0-alpha-dev )
Sep 23 14:04:02.000 [warn] Bug: /usr/bin/tor(main+0x19) [0x55e9e15cb589] (on Tor 0.3.5.0-alpha-dev )
Sep 23 14:04:02.000 [warn] Bug: /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf1) [0x7f246b1fb2e1] (on Tor 0.3.5.0-alpha-dev )
Sep 23 14:04:02.000 [warn] Bug: /usr/bin/tor(_start+0x2a) [0x55e9e15cb5da] (on Tor 0.3.5.0-alpha-dev )It reported that stack trace over and over for few hours, during the time of some internet connectivity problems. The v3 hidden service was unavailable during all this time of course.
Some additional info: The internet was not working properly when this happened. The IPv4 bgp session was down and only IPv6 was working on the server. I have checked the Guard this particular Tor instance was using and it had an IPv6 ORPort (maybe this lead to a false positive wrt to testing our own network connection).
The bigger problem is that when everything came back online, the bug stack trace was not printing in the log file any more, but the v3 onion service did not recover by itself and still wasn't accessible. Had to restart the Tor process entirely and it came back online.