Trac issueshttps://gitlab.torproject.org/legacy/trac/-/issues2020-06-13T17:02:28Zhttps://gitlab.torproject.org/legacy/trac/-/issues/34437migrate help.tpo into a gitlab wiki2020-06-13T17:02:28Zanarcatmigrate help.tpo into a gitlab wikiwe are currently using ikiwiki to host our documentation. that has served us well so far: it's available as a static site in the static mirror system and allows all sysadmins to have a static, offsite copy of the documentation when every...we are currently using ikiwiki to host our documentation. that has served us well so far: it's available as a static site in the static mirror system and allows all sysadmins to have a static, offsite copy of the documentation when everything is down.
but ikiwiki is showing its age. it's an old program written in Perl, difficult to theme and not very welcoming to new users. for example, it's impossible for a user unfamiliar with git to contribute to the documentation. it also has its own unique Markdown dialect that is not used anywhere else. and while Markdown itself is not standardized and has lots of such dialects, there is /some/ convergence around CommonMark and GFM (GitHub's markdown) as de-facto standards at least, which ikiwiki still has to catchup with. it also has powerful macros which are nice to make complex websites, but do not render in the offline documentation, making us dependent on the rendered copy (as opposed to setting up client-side tools to peruse the documentation).
gitlab wikis, in contrast, have a web interface to edit pages. it doesn't have the macros ikiwiki has, but that's nothing a few commandline hacks can't fix... or at least we should consider it. they don't have macros or any more powerful features that ikiwiki has, but maybe that's exactly what we want.
this is not blocking the trac to gitlab migration, but it would be nice to jump onboard with everyone, since we will be migrating the Trac wiki onto gitlab as well...anarcatanarcathttps://gitlab.torproject.org/legacy/trac/-/issues/34373redirect trac.torproject.org to gitlab.torproject.org on dec 12th 20202020-06-13T17:02:22Zanarcatredirect trac.torproject.org to gitlab.torproject.org on dec 12th 2020Once trac is shutdown, redirections from trac.torproject.org needs to be perform to gitlab.torproject.org.
This happens 6 months after trac is put readonly (which is on june 12th) so december 12th 2020.Once trac is shutdown, redirections from trac.torproject.org needs to be perform to gitlab.torproject.org.
This happens 6 months after trac is put readonly (which is on june 12th) so december 12th 2020.Jens KubiezielJens Kubiezielhttps://gitlab.torproject.org/legacy/trac/-/issues/34371make db.torproject.org a real debian archive2020-06-13T17:02:21Zanarcatmake db.torproject.org a real debian archiveI often have trouble uploading packages following our procedure here:
https://help.torproject.org/tsa/howto/build_and_upload_debs/#Uploading_admin_packages
For example, just now I have stumbled upon this:
```
Failed to upload userdir-...I often have trouble uploading packages following our procedure here:
https://help.torproject.org/tsa/howto/build_and_upload_debs/#Uploading_admin_packages
For example, just now I have stumbled upon this:
```
Failed to upload userdir-ldap-cgi_0.3.43~x.tpo.8.dsc to anarcat@alberti.torproject.org:/srv/db.torproject.org/ftp-archive/archive/pool/tpo-all/userdir-ldap-cgi_0.3.43~x.tpo.8.dsc: scp: /srv/db.torproject.org/ftp-archive/archive/pool/tpo-all/userdir-ldap-cgi_0.3.43~x.tpo.8.dsc: Permission denied
```
That was because there was already a `.8.dsc` file from a previous ("UNRELEASED") upload. (I feel it was a mistake to upload such a package in the first place, but that's besides the point: this is only one of many ways this procedure can fail on upload.)
The archive also manually handles OpenPGP certifications and rotations, which is sub-optimal, to say the least, from a security perspective.
Instead, we should use well-known software like reprepro or else to manage the repository, with a proper "incoming" queue.https://gitlab.torproject.org/legacy/trac/-/issues/34317Regading adding a Doc link in "Using Tor with …"2020-06-13T17:37:34ZTracRegading adding a Doc link in "Using Tor with …"I would like you guys to add this following link to the Doc in "Using Tor with …" list. It's a guide on Tor Passive FTP Hidden Service.
Link to Onionized FTP Hidden Service
https://trac.torproject.org/projects/tor/wiki/doc/OnionizeHOWT...I would like you guys to add this following link to the Doc in "Using Tor with …" list. It's a guide on Tor Passive FTP Hidden Service.
Link to Onionized FTP Hidden Service
https://trac.torproject.org/projects/tor/wiki/doc/OnionizeHOWTO/ftp_tor_service
**Trac**:
**Username**: samsapi01GusGushttps://gitlab.torproject.org/legacy/trac/-/issues/34297Explore different options other than crontab to have a flexible scheduling sy...2020-06-13T17:56:18ZBarkin SimsekExplore different options other than crontab to have a flexible scheduling systemCurrently, I use crontab for fetching web pages at regular intervals. I believe it is not the best option, and it is worth exploring the options available.Currently, I use crontab for fetching web pages at regular intervals. I believe it is not the best option, and it is worth exploring the options available.https://gitlab.torproject.org/legacy/trac/-/issues/34290Enhance the available visualizations on the dashboard2020-06-13T17:56:15ZBarkin SimsekEnhance the available visualizations on the dashboardThe visualizations are not easy to understand at first sight. Make them as simple and as meaningful possible.The visualizations are not easy to understand at first sight. Make them as simple and as meaningful possible.https://gitlab.torproject.org/legacy/trac/-/issues/34120Pluggable Transport Versus Bridge2020-06-16T01:12:55ZTracPluggable Transport Versus BridgeIn the Tor preferences>Tor, it states that one can select a Bridge. If so done, then the first option is to "Select a built-in bridge". However, it doesn't. It actually selects a pluggable transport.
Pluggable transports use bridges,...In the Tor preferences>Tor, it states that one can select a Bridge. If so done, then the first option is to "Select a built-in bridge". However, it doesn't. It actually selects a pluggable transport.
Pluggable transports use bridges, but they are not bridges.
Recommend rewording to be more accurate.
**Trac**:
**Username**: TormanToohttps://gitlab.torproject.org/legacy/trac/-/issues/34062Gracefully shutdown services in GetTor2020-06-21T18:06:11ZCecylia BocovichGracefully shutdown services in GetTorI get these errors when I shutdown or restart GetTor. Seems like something we can easily implement:
```
2020-04-29T15:24:03+0000 [gettor#debug] SERVICE:: Calling shutdown on sendmail
2020-04-29T15:24:03+0000 [twisted.internet.defer#crit...I get these errors when I shutdown or restart GetTor. Seems like something we can easily implement:
```
2020-04-29T15:24:03+0000 [gettor#debug] SERVICE:: Calling shutdown on sendmail
2020-04-29T15:24:03+0000 [twisted.internet.defer#critical] Unhandled error in Deferred:
2020-04-29T15:24:03+0000 [twisted.internet.defer#critical]
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/twisted/internet/base.py", line 428, in fireEvent
result = callable(*args, **kwargs)
File "/usr/lib/python3/dist-packages/twisted/application/service.py", line 296, in stopServ
ice
l.append(defer.maybeDeferred(service.stopService))
File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 151, in maybeDeferred
result = f(*args, **kw)
File "/usr/lib/python3/dist-packages/twisted/application/service.py", line 296, in stopServ
ice
l.append(defer.maybeDeferred(service.stopService))
--- <exception caught here> ---
File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 151, in maybeDeferred
result = f(*args, **kw)
File "/srv/gettor.torproject.org/home/gettor/gettor/services/__init__.py", line 60, in stop
Service
self.instance.shutdown()
builtins.AttributeError: 'Sendmail' object has no attribute 'shutdown'
```https://gitlab.torproject.org/legacy/trac/-/issues/34061Reduce amount of GetTor logging2020-06-21T18:06:11ZCecylia BocovichReduce amount of GetTor loggingWe're logging at a very high level (looks like at debug), and outputting frequent successes when we only really need to be logging errors.
For example: a successfully processed email in `log/email_parser.log` outputs:
```
2020-04-27 23:...We're logging at a very high level (looks like at debug), and outputting frequent successes when we only really need to be logging errors.
For example: a successfully processed email in `log/email_parser.log` outputs:
```
2020-04-27 23:18:53+0000 [-] Log opened.
2020-04-27 23:18:53+0000 [process email] New email request received.
2020-04-27 23:18:53+0000 [process email] Reading new email.
2020-04-27 23:18:53+0000 [-] Database query executed successfully.
2020-04-27 23:18:53+0000 [email parser] Building email message from string.
2020-04-27 23:18:53+0000 [email parser] Normalizing and validating FROM email address.
2020-04-27 23:18:53+0000 [email parser] Email address normalized and validated.
2020-04-27 23:18:53+0000 [email parser] Request from [hid]
2020-04-27 23:18:53+0000 [email parser] Found request for links.
2020-04-27 23:18:53+0000 [-] Database query executed successfully.
2020-04-27 23:18:53+0000 [-] Main loop terminated.
2020-04-27 23:18:53+0000 [process email] Email request processed.
```
and in `log/gettor.log`:
```
2020-04-29T14:46:51+0000 [gettor#info] Getting links for windows is.
2020-04-29T14:46:51+0000 [-] Database query executed successfully.
2020-04-29T14:46:51+0000 [gettor#info] Sending links to [hid].
2020-04-29T14:46:51+0000 [gettor#debug] Creating plain text email
2020-04-29T14:46:51+0000 [gettor#debug] Calling asynchronous sendmail.
2020-04-29T14:46:51+0000 [twisted.mail.smtp.ESMTPSenderFactory#info] Starting factory <twisted.mail.smtp.ESMTPSenderFactory object at 0x7f0bba74b780>
2020-04-29T14:46:51+0000 [gettor#info] Email sent successfully.
2020-04-29T14:46:51+0000 [twisted.mail.smtp.ESMTPSenderFactory#info] Stopping factory <twisted.mail.smtp.ESMTPSenderFactory object at 0x7f0bba74b780>
2020-04-29T14:46:51+0000 [-] Database query executed successfully.
2020-04-29T14:46:51+0000 [-] Database query executed successfully.
```
We could reduce this to one log message at most. Especially since this information *should* be captured in the stats database.Cecylia BocovichCecylia Bocovichhttps://gitlab.torproject.org/legacy/trac/-/issues/34010Use io_uring when available2020-06-13T15:53:16ZTracUse io_uring when availableIo_uring is a new subsystem for asynchronous transfers of network or disk/storage i/o.
https://lwn.net/Articles/810414/
It has a great potential in handling concurrent connections and transfers over multiple sockets or disk files.
Sa...Io_uring is a new subsystem for asynchronous transfers of network or disk/storage i/o.
https://lwn.net/Articles/810414/
It has a great potential in handling concurrent connections and transfers over multiple sockets or disk files.
Samba has implemented this on disk io as side, and ib my home / nas setting, it almost doubled total throughput on concurrent reads.
In network situations, it is said to be able to scale to 3-5x performance.
Liburing is a library to be able to utilise this subsystem. I think that Tor really should look at io_uring due to the massive concurrency of a relay.
In my own experience running a relay on a low end hardware for two years and the low end hardware was never able to fill the fiber connection. It seems to be quite a lot of internal overhead, perhaps io_uring could really help.
**References**
https://lwn.net/Articles/810414/
https://lwn.net/Articles/776428/
https://git.kernel.dk/cgit/liburing/
https://github.com/axboe/liburing
**Other projects using io_uring**
https://wiki.samba.org/index.php/Samba_4.12_Features_added/changed#.27io_uring.27_vfs_module
https://github.com/ceph/ceph/pull/27392
**Trac**:
**Username**: torryTor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/33949python 2 end of life coordination2020-06-13T17:01:47Zanarcatpython 2 end of life coordinationPython 2.7.18 has just been released. It is the last Python 2 release that will ever happen, and Python 2 is now unsupported, end of life, [dead](https://www.enricozini.org/blog/2020/python/python-2-is-dead/).
It is likely that the next...Python 2.7.18 has just been released. It is the last Python 2 release that will ever happen, and Python 2 is now unsupported, end of life, [dead](https://www.enricozini.org/blog/2020/python/python-2-is-dead/).
It is likely that the next Debian release (bullseye) will not support Python 2 at all. It's also possible the current release (buster) does not support Python 2 for security issues forever. So we have *some* time, in practice, to handle this problem. But we definitely will need to finish this migration before some time around 2022, and the sooner the better.
Until then, we need to figure out a strategy on how to handle that transition. Some of our code has been written for Python 3, but we have a large amount of Python-2-only code that is running, in multiple places. Some of it is TPA's responsibility, but other code is ran by teams or service admins.
Since we run stretch or buster everywhere, we're in a good position to *not* have to support both Python 2 and Python 3 at once: we can just *migrate* to python 3. Stretch has Python 3.5 so we could target that as a minimum version. But we could also assume we will have completed the Buster upgrade by then and just target the more featureful Python 3.7.
In any case, we need a plan for this and it would be wise to do it before we're backed into a corner.
Some resources:
* http://python3porting.com/ - python 3 porting book, freely available
* https://python3statement.org/practicalities/ - some more advice on porting
* https://docs.python.org/3/howto/pyporting.html - upstream guide, which still recommends supporting python 2.7https://gitlab.torproject.org/legacy/trac/-/issues/33943bandwidth-shaping script link Path not found2020-06-13T17:36:53Zcypherpunksbandwidth-shaping script link Path not foundon page:
https://support.torproject.org/operators/bandwidth-shaping/
The link from last script, leads to nowhere.
https://gitweb.torproject.org/tor.git/tree/contrib/operator-tools/linux-tor-prio.sh
Path not foundon page:
https://support.torproject.org/operators/bandwidth-shaping/
The link from last script, leads to nowhere.
https://gitweb.torproject.org/tor.git/tree/contrib/operator-tools/linux-tor-prio.sh
Path not foundGusGushttps://gitlab.torproject.org/legacy/trac/-/issues/33942Node priority info2020-06-13T17:36:52ZcypherpunksNode priority infoIs it true, that TOR prioritizes high speed nodes over slow-ones? I could not find any info in FAQ on this.
And its not possible to see how much traffic goes thru prioritized nodes compared to overall network on TOR Metrics. How much % ...Is it true, that TOR prioritizes high speed nodes over slow-ones? I could not find any info in FAQ on this.
And its not possible to see how much traffic goes thru prioritized nodes compared to overall network on TOR Metrics. How much % of requests do they serve?
Im sorry if i spelled something wrong, english is not my main language.GusGushttps://gitlab.torproject.org/legacy/trac/-/issues/33921gitlab monitoring2020-06-13T17:01:44Zanarcatgitlab monitoringwe need to have gitlab metrics in grafana. we also need to make sure things work (nagios).
this implies monitoring the webserver (nginx i guess?) and it would also be nice to have internal gitlab metrics. there's a builtin prometheus ex...we need to have gitlab metrics in grafana. we also need to make sure things work (nagios).
this implies monitoring the webserver (nginx i guess?) and it would also be nice to have internal gitlab metrics. there's a builtin prometheus exporter in gitlab so we should be able to reuse that. see:
https://docs.gitlab.com/ee/administration/monitoring/prometheus/
https://docs.gitlab.com/ee/administration/monitoring/prometheus/gitlab_metrics.htmlHiroHirohttps://gitlab.torproject.org/legacy/trac/-/issues/33920Remove errors or typos from TorNet.py and Templating.py2020-06-13T13:31:50ZMrSquancheeRemove errors or typos from TorNet.py and Templating.pyI use vscode for programming and when I open chutney files, TorNet.py
and Templating.py, vscode reports a bunch of errors and warnings.
These are often due to typing mistakes and unused variables.
This ticket aims at removing them.I use vscode for programming and when I open chutney files, TorNet.py
and Templating.py, vscode reports a bunch of errors and warnings.
These are often due to typing mistakes and unused variables.
This ticket aims at removing them.MrSquancheeMrSquancheehttps://gitlab.torproject.org/legacy/trac/-/issues/33876Back Button on Tor Browser Android Closes App When Log is Open2020-06-16T01:12:29ZTracBack Button on Tor Browser Android Closes App When Log is OpenThis is a small UX quirk that can frustrate/confuse new users, especially those who are unfamiliar with Tor and are curious about the log. When you open the log in Tor Browser on Android it fills up the entire screen and if the user pres...This is a small UX quirk that can frustrate/confuse new users, especially those who are unfamiliar with Tor and are curious about the log. When you open the log in Tor Browser on Android it fills up the entire screen and if the user presses the back button on their device the app will close. Generally when a screen is filled on Android pressing back takes you to the previous screen. I think little things could instill distrust in newer users.
This issue used to exist in Orbot a few years ago https://github.com/guardianproject/orbot/pull/139
I apologize if this isn't the correct spot to report this. I'm new to trac, and am open to feedback if there's a better spot to file this :) Also, I've implemented a fix and I'm still figuring out where to push my code and put it up for review... <3
**Trac**:
**Username**: bimhttps://gitlab.torproject.org/legacy/trac/-/issues/33868fabric (incorrectly) asumes User root ssh_config2020-06-13T17:01:37Zanarcatfabric (incorrectly) asumes User root ssh_configour fabric code assumes we have a `User root` block for all tpo hosts, which is incorrect: i actually deliberately set `User anarcat` on pauli, for example, so that I don't push as root.
this should be fixed with a fabric-specific config.our fabric code assumes we have a `User root` block for all tpo hosts, which is incorrect: i actually deliberately set `User anarcat` on pauli, for example, so that I don't push as root.
this should be fixed with a fabric-specific config.anarcatanarcathttps://gitlab.torproject.org/legacy/trac/-/issues/33839Correct 'was not internal' to 'was internal' in test_external_ip()2020-06-13T15:52:57ZTracCorrect 'was not internal' to 'was internal' in test_external_ip()source: https://github.com/torproject/tor/pull/1856
Fixing a typo in a string that is printed on event of a test failing, in `src/test/test_addr.c`.
**Trac**:
**Username**: kimimarosource: https://github.com/torproject/tor/pull/1856
Fixing a typo in a string that is printed on event of a test failing, in `src/test/test_addr.c`.
**Trac**:
**Username**: kimimaroTor: 0.4.3.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/33810ganeti monitoring2020-06-13T17:01:33Zanarcatganeti monitoringwe're migrating everything into ganeti, but maybe there's some extra monitoring we could think about, as ganeti is way more knowledgeable about its own internals than libvirt was. or at least that's the feeling I get.
some ideas:
* we...we're migrating everything into ganeti, but maybe there's some extra monitoring we could think about, as ganeti is way more knowledgeable about its own internals than libvirt was. or at least that's the feeling I get.
some ideas:
* we could have a nagios plugin that checks for N+1. riseup has something like this
* we could have a grafana dashboard that shows us the state of the cluster. we already have the main dashboard which we can set to show only the ganeti cluster
The current memory view goes about like this:
![snap-2020.04.03-17.33.23.png,700](uploads/snap-2020.04.03-17.33.23.png,700)
I'm not sure how we could improve this, but it seems to me having global (and/or per node?) memory, CPU, network and disk usage would be a great improvement as well.anarcatanarcathttps://gitlab.torproject.org/legacy/trac/-/issues/33798Please destroy static site atlas.torproject.org2020-06-13T17:01:33ZirlPlease destroy static site atlas.torproject.orgTo give you a chance to exercise the procedure at: https://help.torproject.org/tsa/howto/static-component/
In #25283 we have done an analysis to show that most things are no longer hitting this site, and we don't want to keep things aro...To give you a chance to exercise the procedure at: https://help.torproject.org/tsa/howto/static-component/
In #25283 we have done an analysis to show that most things are no longer hitting this site, and we don't want to keep things around forever.
This is not urgent, no need to rush.anarcatanarcat