Onion Service dies randomly after ~1-5 days
Summary
Creating an Onion Service with Stem seems stays up randomly for approximately 1-5 days, it then goes permanently down.
Steps to reproduce:
- Use Tor+Stem+Flask to start an ephemeral Onion Service web server
- Wait up to 5 days
- See that at some point it's no longer possible to connect to the Onion Service
I wrote a script that reproduces the issue, but as per the current discussion on tor-talk
mailing list [Tor users from Finland jumped from 25 000 to 200 000], the script may have had an unintentional side-effect of creating new users, so running the script as it is, may mess up metrics graphs. Thus it definitely needs review (and most likely revision) before use.
What is the current bug behavior?
Onion Service goes down unpredictably and does not recover.
What is the expected behavior?
Onion Service stays up until it's turned off by the host.
Environment
OS
- Ubuntu 21.04
- Lubuntu 21.04
Tor
- Tor version: 0.4.5.6
- Tor install method: APT
Stem
- Stem version: 1.8.0
- Stem install method: PIP
Relevant logs and/or screenshots:
Below are the logs from two instances of the bug-reproducing script. As the delay before the service goes permanently down is random, I had the script collect data about the server up-times. Note that the timestamp of Main: Client reports
line is intentionally always one hour after the associated server last seen
timestamp.
The two interesting things here are, the Onion Service stays up randomly for 1..5 days. Since it's more or less random, it's hard to be sure about the range. The more interesting thing here is the "server last seen" timestamps seem to be often on the hour, but not always (I didn't test against general online connectivity so it's hard to say if those anomalies are related to DSL availability)
Test instance 1
18-12-21 22:02:02 - Main: Starting bug testing.
20-12-21 01:00:52 - Main: Client reports
Server first seen : 18-12-21 22:02:10
Server last seen : 20-12-21 00:00:00
Server's been down for : 01h 00m 37.7s
Restarting with new Onion Service
21-12-21 04:02:10 - Main: Client reports
Server first seen : 20-12-21 01:01:02
Server last seen : 21-12-21 03:00:02
Server's been down for : 01h 00m 06.6s
Restarting with new Onion Service
26-12-21 16:01:15 - Main: Client reports
Server first seen : 21-12-21 04:02:18
Server last seen : 26-12-21 15:00:00
Server's been down for : 01h 00m 02.3s
Restarting with new Onion Service
27-12-21 18:42:49 - Main: Client reports
Server first seen : 26-12-21 16:01:22
Server last seen : 27-12-21 17:40:43
Server's been down for : 01h 00m 04.2s
Restarting with new Onion Service
28-12-21 22:01:05 - Main: Client reports
Server first seen : 27-12-21 18:42:56
Server last seen : 28-12-21 21:00:00
Server's been down for : 01h 00m 01.6s
Restarting with new Onion Service
30-12-21 01:01:20 - Main: Client reports
Server first seen : 28-12-21 22:01:11
Server last seen : 30-12-21 00:00:00
Server's been down for : 01h 00m 02.7s
Restarting with new Onion Service
31-12-21 03:18:37 - Main: Client reports
Server first seen : 30-12-21 01:01:27
Server last seen : 31-12-21 02:18:24
Server's been down for : 01h 00m 06.3s
Restarting with new Onion Service
05-01-22 17:01:00 - Main: Client reports
Server first seen : 31-12-21 03:18:55
Server last seen : 05-01-22 16:00:00
Server's been down for : 01h 00m 04.2s
Restarting with new Onion Service
06-01-22 20:00:58 - Main: Client reports
Server first seen : 05-01-22 17:01:07
Server last seen : 06-01-22 19:00:01
Server's been down for : 01h 00m 21.0s
Restarting with new Onion Service
08-01-22 00:00:33 - Main: Client reports
Server first seen : 06-01-22 20:01:15
Server last seen : 07-01-22 23:00:00
Server's been down for : 01h 00m 16.7s
Restarting with new Onion Service
09-01-22 03:00:56 - Main: Client reports
Server first seen : 08-01-22 00:00:45
Server last seen : 09-01-22 02:00:01
Server's been down for : 01h 00m 03.4s
Restarting with new Onion Service
Test instance 2
18-12-21 22:02:03 - Main: Starting bug testing.
20-12-21 01:00:24 - Main: Client reports
Server first seen : 18-12-21 22:02:14
Server last seen : 20-12-21 00:00:00
Server's been down for : 01h 00m 03.1s
Restarting with new Onion Service
21-12-21 04:00:45 - Main: Client reports
Server first seen : 20-12-21 01:00:33
Server last seen : 21-12-21 03:00:01
Server's been down for : 01h 00m 02.7s
Restarting with new Onion Service
26-12-21 16:00:41 - Main: Client reports
Server first seen : 21-12-21 04:00:57
Server last seen : 26-12-21 15:00:01
Server's been down for : 01h 00m 06.1s
Restarting with new Onion Service
27-12-21 19:17:31 - Main: Client reports
Server first seen : 26-12-21 16:00:51
Server last seen : 27-12-21 17:57:09
Server's been down for : 01h 01m 40.2s
Restarting with new Onion Service
28-12-21 23:00:48 - Main: Client reports
Server first seen : 27-12-21 19:17:41
Server last seen : 28-12-21 22:00:00
Server's been down for : 01h 00m 02.2s
Restarting with new Onion Service
30-12-21 02:02:41 - Main: Client reports
Server first seen : 28-12-21 23:00:57
Server last seen : 30-12-21 01:00:01
Server's been down for : 01h 01m 32.1s
Restarting with new Onion Service
31-12-21 17:14:59 - Main: Client reports
Server first seen : 30-12-21 02:02:50
Server last seen : 31-12-21 16:13:50
Server's been down for : 01h 00m 55.9s
Restarting with new Onion Service
01-01-22 20:01:21 - Main: Client reports
Server first seen : 31-12-21 17:15:31
Server last seen : 01-01-22 19:00:02
Server's been down for : 01h 00m 25.6s
Restarting with new Onion Service
02-01-22 23:01:32 - Main: Client reports
Server first seen : 01-01-22 20:01:54
Server last seen : 02-01-22 22:00:03
Server's been down for : 01h 00m 46.2s
Restarting with new Onion Service
04-01-22 02:01:17 - Main: Client reports
Server first seen : 02-01-22 23:01:44
Server last seen : 04-01-22 01:00:00
Server's been down for : 01h 00m 27.1s
Restarting with new Onion Service
05-01-22 05:01:01 - Main: Client reports
Server first seen : 04-01-22 02:01:31
Server last seen : 05-01-22 04:00:00
Server's been down for : 01h 00m 24.5s
Restarting with new Onion Service
10-01-22 17:02:04 - Main: Client reports
Server first seen : 05-01-22 05:01:12
Server last seen : 10-01-22 16:00:03
Server's been down for : 01h 00m 15.1s
Restarting with new Onion Service
11-01-22 20:01:28 - Main: Client reports
Server first seen : 10-01-22 17:02:16
Server last seen : 11-01-22 19:00:00
Server's been down for : 01h 01m 04.7s
Restarting with new Onion Service
12-01-22 23:02:50 - Main: Client reports
Server first seen : 11-01-22 20:02:08
Server last seen : 12-01-22 22:00:40
Server's been down for : 01h 00m 49.0s
Restarting with new Onion Service
Possible fixes:
No idea, this might be a bug with Tor core, or it might have something to do with Stem.