Number of Introduction Points for a (SingleOnion?) HS seems variable, or degrades with time
I'm helping some people tune a very large Tor Onion site which has recently converted to 2.9.9 and Single Onions.
There have been some strange reachability issues which I am helping triage.
Analysis is tricky because independent copies of the onion exist in two different datacentres, and they both descriptors into the HSDirs.
To date this configuration has not contributed to any outages that we're aware.
For this ticket the specific onion address is "fbsbx2q4mvcl63pw" - though at least one other onion address (out of 3) has been showing the same symptoms.
All onions are configured with "HiddenServiceNumIntroductionPoints 10"
I have been monitoring the Onion descriptors with a Stem script* that prints the HS Descriptor IPs, and over time the listed IPs appear to drop from the expected 10, to as low as 4.
Partial output from the script log, for this morning, is at:
https://gist.github.com/alecmuffett/2910328730012f9af3410a28e5e6071e
At 0800h the IPs in retrieved HSDescriptor (reminder: from one of two purposely-duplicated HS instances) looked like this, lines 1..12 in the Gist; note NumIntroductionPoints/nip=10
Mon 13 Feb 2017 08:00:55 GMT v=2 age=55 nip=10 pub(2017-02-13 08:00:00) now(2017-02-13 08:00:55) fbsbx2q4mvcl63pw 0: 134.19.177.109:443 1: 178.254.20.134:9001 2: 185.73.220.8:443 3: 188.114.140.245:9001 4: 37.187.4.8:21 5: 46.101.102.71:443 6: 46.4.49.201:9001 7: 62.210.82.44:21 8: 62.210.92.11:9001 9: 83.44.207.3:9001
Yet further down the file, a different descriptor, possibly from the other instance (lines 50-57)
Mon 13 Feb 2017 08:21:03 GMT v=2 age=1263 nip=6 pub(2017-02-13 08:00:00) now(2017-02-13 08:21:03) fbsbx2q4mvcl63pw 0: 188.118.217.236:443 1: 192.42.116.161:9001 2: 37.187.4.8:21 3: 80.248.208.131:9001 4: 82.165.207.102:9001 5: 88.190.210.240:9001
...only 6 introduction points.
If we assume (?) that it's unlikely for the two daemons to pick the same introduction point, the fact that "188.118.217.236:443" is present in all the descriptors in lines 40..150 suggests that all these descriptors are from the same daemon instance - however the number of introduction points varies.
See attached image for highlight. Is this expected?
*script: https://github.com/alecmuffett/halfagigonion/blob/master/ls-hsdir.py