Trac issueshttps://gitlab.torproject.org/legacy/trac/-/issues2020-06-13T18:04:33Zhttps://gitlab.torproject.org/legacy/trac/-/issues/34141Stop generating .tpf files2020-06-13T18:04:33ZKarsten LoesingStop generating .tpf filesAs of last week, CollecTor, metrics-lib, and metrics-web all know how to process OnionPerf's .json files. Unless anything else uses .tpf files we can now stop generating those.
Suggested process:
- Discuss any remaining uses of .tpf fi...As of last week, CollecTor, metrics-lib, and metrics-web all know how to process OnionPerf's .json files. Unless anything else uses .tpf files we can now stop generating those.
Suggested process:
- Discuss any remaining uses of .tpf files where .json files are not yet supported.
- Stop generating .tpf files on deployed instances op-{hk,nl,us}2, and if somebody complains, re-generate them from existing log files until they have switched.
- Remove the code that generates .tpf files from OnionPerf.https://gitlab.torproject.org/legacy/trac/-/issues/34030Indexer ignores a file after moving it away and back shortly after2020-06-13T17:52:31ZKarsten LoesingIndexer ignores a file after moving it away and back shortly afterToday I tried to trigger the new Nagios CollecTor check by moving away a recent Snowflake file and moving it back a few minutes later.
The first part of moving the file away worked fine to the effect that it was not included in the `ind...Today I tried to trigger the new Nagios CollecTor check by moving away a recent Snowflake file and moving it back a few minutes later.
The first part of moving the file away worked fine to the effect that it was not included in the `index.json` file anymore.
However, the second part of moving the file back did not cause the indexer to include the file in the `index.json` file again.
What I had to do was touch the file and setting a slightly different last-modified timestamp. More precisely, changing the second was not sufficient, but changing the minute was.
I haven't looked at the code yet, but I could imagine that it's related to some optimization we did about not indexing files that we had already indexed before. Maybe it's also related to the way how we keep files available for a certain time after they dropped out of the `index.json` file. Or maybe it's caused by both.
This is not a critical bug, but it's also likely not a complex fix. It would be good to fix this, because it would probably confuse another CollecTor operator who didn't happen to write the indexer code.https://gitlab.torproject.org/legacy/trac/-/issues/34029Add more command-line arguments to the Nagios CollecTor check script2020-06-13T17:52:30ZKarsten LoesingAdd more command-line arguments to the Nagios CollecTor check scriptMoved here from #33972:
_I'll also look into the parameters and using argparse next week. Unfortunately, the check wouldn't work for corsicum right now anyway, because that CollecTor instance does not archive all descriptor types. It wo...Moved here from #33972:
_I'll also look into the parameters and using argparse next week. Unfortunately, the check wouldn't work for corsicum right now anyway, because that CollecTor instance does not archive all descriptor types. It would just keep shouting about timestamps being missing. Maybe we'll need to add another option to only complain about outdated timestamp, not about missing timestamps. Added to my list._
These are two changes:
- add two separate host and IP parameters as suggested on the other ticket and
- add another parameter for ignoring missing timestamps.
None of these changes are critical, but making them sooner rather than later reduces the overhead for context switching.
The check script is [here](https://gitweb.torproject.org/admin/tor-nagios.git/tree/tor-nagios-checks/checks/tor-check-collector).https://gitlab.torproject.org/legacy/trac/-/issues/34018Remove op-ab from CollecTor configuration2020-06-13T17:52:30ZirlRemove op-ab from CollecTor configurationThis is going away.This is going away.https://gitlab.torproject.org/legacy/trac/-/issues/33933Bandwidth distribution graph has "0 Gbit/s" for all y axis labels2020-06-13T18:15:34ZRoger DingledineBandwidth distribution graph has "0 Gbit/s" for all y axis labelsSee e.g. https://metrics.torproject.org/advbwdist-perc.html?start=2015-01-19&end=2020-04-18&p=50
This is likely the same type of rounding bug as #33066.
(I wonder if there are other graphs that have this issue, too?)See e.g. https://metrics.torproject.org/advbwdist-perc.html?start=2015-01-19&end=2020-04-18&p=50
This is likely the same type of rounding bug as #33066.
(I wonder if there are other graphs that have this issue, too?)https://gitlab.torproject.org/legacy/trac/-/issues/33888Release ExoneraTor 4.4.02020-06-13T17:55:23ZKarsten LoesingRelease ExoneraTor 4.4.0I'm going to put out a new ExoneraTor release today.I'm going to put out a new ExoneraTor release today.https://gitlab.torproject.org/legacy/trac/-/issues/33887Release metrics-lib 2.11.02020-06-13T17:58:26ZKarsten LoesingRelease metrics-lib 2.11.0I'm going to put out a new metrics-lib release now.I'm going to put out a new metrics-lib release now.https://gitlab.torproject.org/legacy/trac/-/issues/33754I discovered a Tor node using TCP port 9999 (service: "distinct" / "abyss"). ...2020-06-13T17:54:39ZTracI discovered a Tor node using TCP port 9999 (service: "distinct" / "abyss"). Is this normal?I recently ran Netstat, a program within Network Utility (I did this while using Tor Browser). When the option "Display the state of all current socket connections" was selected, two Tor IP addresses appeared in the "Foreign address" col...I recently ran Netstat, a program within Network Utility (I did this while using Tor Browser). When the option "Display the state of all current socket connections" was selected, two Tor IP addresses appeared in the "Foreign address" column. One IP address is the Tor entry node I am currently using, and the other IP address is an IP address using TCP port 9999 -- the "distinct" service, also known as "abyss". What I'm wondering is, is it normal for a Tor node to use TCP port 9999 ("distinct") while using Tor? Is this a sign of malicious activity?
I did a terminal command to see if I could find out more information about the IP address using port 9999, and it said it is using tor.real.
In addition, the mysterious Tor IP address appears to be using the following ports on my computer: 22, 80, 110, 143, 443, 993, 995, 9998 and 9999.
The reason I know the mysterious IP address is a Tor IP address is because Terminal told me it is using tor.real, and Tor Exonerator confirmed that it is a Tor IP address.
**Trac**:
**Username**: Tor235https://gitlab.torproject.org/legacy/trac/-/issues/33720Check exit scanner in Nagios2020-06-13T17:48:46ZirlCheck exit scanner in Nagioshttps://gitlab.torproject.org/legacy/trac/-/issues/33719Check DNSEL in Nagios2020-06-13T17:48:46ZirlCheck DNSEL in Nagioshttps://gitlab.torproject.org/legacy/trac/-/issues/33718Check check.torproject.org in Nagios2020-06-13T17:48:45ZirlCheck check.torproject.org in Nagioshttps://gitlab.torproject.org/legacy/trac/-/issues/33714Use Travis CI for ansible-lint2020-06-13T17:48:43ZirlUse Travis CI for ansible-lintWe are no longer using the GitLab CI service, but we do still need CI, so let's move it to Travis CI and use GitHub integration for this.We are no longer using the GitLab CI service, but we do still need CI, so let's move it to Travis CI and use GitHub integration for this.https://gitlab.torproject.org/legacy/trac/-/issues/33713Use Travis CI for cfn-lint2020-06-13T17:48:43ZirlUse Travis CI for cfn-lintWe are no longer using the GitLab CI service, but we do still need CI, so let's move it to Travis CI and use GitHub integration for this.We are no longer using the GitLab CI service, but we do still need CI, so let's move it to Travis CI and use GitHub integration for this.https://gitlab.torproject.org/legacy/trac/-/issues/33663Check checktest.py related errors shown by our network-health scanners2020-11-13T13:45:10ZGeorg KoppenCheck checktest.py related errors shown by our network-health scannersI often see something like
```
2020-03-17 21:03:56,973 modules.checktest [ERROR] Check thinks <https://metrics.torproject.org/rs.html#details/296B2178FD742AB35AB20C9ADF04D5DFD3D407EB> isn't Tor. Desc addr is 206.55.74.0 and check addr i...I often see something like
```
2020-03-17 21:03:56,973 modules.checktest [ERROR] Check thinks <https://metrics.torproject.org/rs.html#details/296B2178FD742AB35AB20C9ADF04D5DFD3D407EB> isn't Tor. Desc addr is 206.55.74.0 and check addr is 206.55.74.0.
```
. We should figure out a) what's up with that and b) whether we actually still need that test to be running.https://gitlab.torproject.org/legacy/trac/-/issues/33655Make all server-side modules independent of server locale or time zone2020-06-13T18:10:36ZKarsten LoesingMake all server-side modules independent of server locale or time zoneIn #24532 we made metrics-web's daily updater runs independent of server locale and time zone. We also said that if the fix looks good, we should apply the same fix to all other `main` methods in our various code bases. This includes:
`...In #24532 we made metrics-web's daily updater runs independent of server locale and time zone. We also said that if the fix looks good, we should apply the same fix to all other `main` methods in our various code bases. This includes:
```
org.torproject.metrics.collector.Main
org.torproject.metrics.exonerator.ExoneraTorDatabaseImporter
org.torproject.metrics.exonerator.ServerMain
org.torproject.metrics.onionoo.cron.Main
org.torproject.metrics.onionoo.server.ServerMain
org.torproject.metrics.web.ServerMain <- already done in #24532
org.torproject.metrics.web.UpdateNews
```
This ticket tracks progress of making the same changes in the remaining 6 places. We could create child tickets if we wanted, but maybe that's overkill.https://gitlab.torproject.org/legacy/trac/-/issues/33634Confusing JavaScript error when entering email address in Relay Search2020-06-13T18:08:30ZGeorg KoppenConfusing JavaScript error when entering email address in Relay SearchIf I enter a relay fingerprint on Relay Search I get all the info about the respective relay as a result, great. Now if I enter an email address in Relay Search I get a confusing JavaScript message:
```
JavaScript Error!
There is a prob...If I enter a relay fingerprint on Relay Search I get all the info about the respective relay as a result, great. Now if I enter an email address in Relay Search I get a confusing JavaScript message:
```
JavaScript Error!
There is a problem with your javascript environment, you may have noscript enabled on the remote onionoo backend. Try temporarily allowing noscript to connect to the backend IP address. If the problem persits consult the bugtracker.
```
Note, I did not change any of my JavaScript settings between entering the fingerprint and the email address: I have it enabled in both scenarios.
Now, ideally when entering the email address (without the "contact:" prefix which makes the whole thing work) that just give me the relay/a list of relays with the respective contact info, just like entering the plain fingerprint does.
I am fine if that's asking too much. :) But then at least the wrong JavaScript error should get replaced with something more meaningful.https://gitlab.torproject.org/legacy/trac/-/issues/33509Confirm that CollecTor will be happy consuming data from the new check service2020-06-13T17:52:29ZirlConfirm that CollecTor will be happy consuming data from the new check serviceThe exit addresses are available, for testing, at:
https://check-01.torproject.org/exit-addresses
This will not have a valid TLS cert though, you can also add check.tpo to /etc/hosts for that IP address and it will then have a valid cert.The exit addresses are available, for testing, at:
https://check-01.torproject.org/exit-addresses
This will not have a valid TLS cert though, you can also add check.tpo to /etc/hosts for that IP address and it will then have a valid cert.https://gitlab.torproject.org/legacy/trac/-/issues/33507Deploy new exit scanner and check combo on check-01 using metrics-cloud2020-06-13T17:54:36ZirlDeploy new exit scanner and check combo on check-01 using metrics-cloudThis is not the same project as #29650, which was a more ambitious attempt at fixing some of the shortcomings of the current setup. Instead, this project aims to keep the service running with minimal changes until we have more time/fundi...This is not the same project as #29650, which was a more ambitious attempt at fixing some of the shortcomings of the current setup. Instead, this project aims to keep the service running with minimal changes until we have more time/funding to take on the increased level of work that would be involved for the more ambitious project. For avoidance of doubt, I still believe that #29650 is going to be needed soon and is entirely valid work for the Metrics Team to do.
This ticket blocks #29399 which is of a high priority to the sysadmin team.
The parts of the existing service that can be reused will be reused, which is really just the check application. The check application is written in Go and uses local filesystem data sources (which have to be updated by a tor client). It has an embedded web server and we reverse proxy this via Apache.
The exit scanner and DNS server were written in Haskell. We can't maintain this so instead we are using exitmap wrapped in a Python script to produce exit lists compatible with the existing format (i.e. we're not going to a new format yet). A second python script runs by cron job to write out a BIND compatible zone file, which is served by bind.
The exact split between Metrics Team and Sysadmin Team responsibilities are details in the ops doc for this service (which is yet to be written).https://gitlab.torproject.org/legacy/trac/-/issues/33497Translation on metrics.torproject.org2020-06-13T18:15:33ZTracTranslation on metrics.torproject.orgI think it is necessary to have different translations of metrics.torproject.org.
There is an old closed ticket https://trac.torproject.org/projects/tor/ticket/20867 but there are still no translations except from https://exonerator.torp...I think it is necessary to have different translations of metrics.torproject.org.
There is an old closed ticket https://trac.torproject.org/projects/tor/ticket/20867 but there are still no translations except from https://exonerator.torproject.org/.
If we could take the content to the 'Tor Project Web' on transifex there would be volunteers for many languages which would translate it.
It's also not that much so i think one could translate the whole content in one day.
**Trac**:
**Username**: kucehttps://gitlab.torproject.org/legacy/trac/-/issues/33493Encourage operators to ignore bridge flags on relay search2020-06-13T18:08:30ZteorEncourage operators to ignore bridge flags on relay searchThe bridge flags on relay search confuse some bridge operators:
https://lists.torproject.org/pipermail/tor-relays/2020-February/018188.html
Can we grey them out, or otherwise indicate that they're not used by clients?
Can we change the...The bridge flags on relay search confuse some bridge operators:
https://lists.torproject.org/pipermail/tor-relays/2020-February/018188.html
Can we grey them out, or otherwise indicate that they're not used by clients?
Can we change the mouseover help message on the "flags" heading for bridges?
Can we also change the definition of Flags, and the Bridge Descriptors definition, so they talk about bridge flags?