install tor-geoipdb package from bullseye-backports in polyanthum
The geoipdb we are using in polyanthum is pretty old, and depends on updates from debian stable (right now oldstable). There are more frequently updated packages for it in deb.torproject.org, could we use those? can unattended-upgrades or something be configured to update tor-geoipdb from deb.tpo?
Designs
Activity
i looked at this quickly and it's a little bizarre: we do not use deb.tpo on any of our TPA machines right now. i guess i could make an exception for this, but would you be happy with the version in stable instead?
how do the versions in stable / testing / unstable compare with the one from deb.tpo?
i'm a little worried about adding that repo because it would enable horizontal movement across the infrastructure (specifically from palmeri/gitlab/the static mirror system to polyanthum), something which we typically try to avoid except in rare occasions where there isn't any other solution (e.g. LDAP, Nagios, Puppet).
for now i did this in Puppet, currently hidden behind a feature flag, in
profile::bridges:# cargo culted with some modifications (package list and pin values) # from tor::install # # the pin value was 1000, we bring it down to 500 so that we respect # version numbers (e.g. don't upgrade if current release is newer) # # the packages list included tor, which we don't need to upgrade # right now if $use_deb_tpo { apt::source { 'torproject': comment => 'TorProject', location => '[signed-by=/usr/share/keyrings/torproject.gpg] https://deb.torproject.org/torproject.org', repos => 'main', release => $tor::upstream_release, require => [ Package['apt-transport-https'] ]; } apt::pin { 'torproject': packages => [ 'tor-geoipdb' ], priority => 500, originator => 'TorProject', component => 'main'; 'torproject-negative': packages => '*', priority => -1, originator => 'TorProject', component => 'main'; } file { '/usr/share/keyrings/torproject.gpg': ensure => present, source => 'puppet:///modules/tor/torproject.gpg', owner => 'root', group => 0, mode => '0644'; } }that should normally take care of upgrades, but maybe unattended-upgrades would need an exception too, which is a little more tricky to implement.
added Needs Information label
assigned to @anarcat
how do the versions in stable / testing / unstable compare with the one from deb.tpo?
We relay on the geoip db to do things like recommending the apropate circumvention configuration in Tor Browser. Having the version from stable means having several years old db. Will be nice to have it a bit fresher.
The bullseye-bakcports repo seems to have a pretty up to date package. Will bullseye-backports be easier to set up than deb.tpo?
I understand that this change will not be done until polyanthum is migrated to bullseye, but hopefully this is very soon.
-
meskio commented:
how do the versions in stable / testing / unstable compare with the one from deb.tpo?
We relay on the geoip db to do things like recommending the apropate circumvention configuration in Tor Browser. Having the version from stable means having several years old db. Will be nice to have it a bit fresher.
Got it.
The bullseye-bakcports repo seems to have a pretty up to date package. Will bullseye-backports be easier to set up than deb.tpo?
Yes, much easier, because we don't need to maintain it ourselves.
I understand that this change will not be done until polyanthum is migrated to bullseye, but hopefully this is very soon.
Hopefully by the end of the month, is that alright?
a.
...
On 2022-05-16 10:51:48, meskio wrote:
-- Antoine Beaupré torproject.org system administration
-
I understand that this change will not be done until polyanthum is migrated to bullseye, but hopefully this is very soon.
Hopefully by the end of the month, is that alright?
Yes, that is alright. We've lived with an outdated geoipdb for a long time, a couple of weeks more are not going to change much.
Thank yo.
changed milestone to %Debian 11 bullseye upgrade
added Next label and removed Needs Information label
mentioned in issue #40692 (closed)
added Anti-Censorship label
removed Anti-Censorship label
added For Anticensorship Team lifecycle labels
added Puppet label
-
bridgestrap needs fixes from the latest version of tor. We used to have a static binary of tor compiled, but it stopped working after the upgrade (tpo/anti-censorship/bridgestrap#34 (closed)). Instead of compiling a new version myself, I think our problems will be solved if we get the latest version of bullseye-backports installed there. Can we not only get tor-geoipdb but also the tor package installed from bullseye-backports in polyanthum?
-
bullseye-backports is actually already configured on polyanthum. do you want me to have those two upgraded already? it won't cover for unattended-upgrades (i think) but it would fix your immediate problem.
a.
...
On 2022-05-20 17:51:28, meskio wrote:
meskio commented:
bridgestrap needs fixes from the latest version of tor. We used to have a static binary of tor compiled, but it stopped working after the upgrade (tpo/anti-censorship/bridgestrap#34 (closed)). Instead of compiling a new version myself, I think our problems will be solved if we get the latest version of bullseye-backports installed there. Can we not only get tor-geoipdb but also the tor package installed from bullseye-backports in polyanthum?
-- Antoine Beaupré torproject.org system administration
-
meanwhile i started working on the backports side of things by refactoring our unattended-upgrades configuration, in a
spinoff-unattended-upgrades-extrasbranch. basically, i stop specifying theoriginfield in there, so that we rely on pinning instead. it's actually an idea suggested by the README file, so it is not that crazy.but i am not deploying this now, on a friday. will need progressive deployment next week.
bridgestrap needs Tor version 0.4.7.1-alpha or later to run properly (because of the new feature introduced in tpo/core/tor#40209 (closed)).
I guess it used to run a manually-patched-from-source Tor, but having a proper 0.4.7 Tor package installed on the system sounds even better. Whether that comes from Debian backports or deb.torproject.org is fine imo.
-
And! Check out this ticket tpo/anti-censorship/bridgestrap#17 (closed) which we'll be able to close once there is an 0.4.7.x deb running on polyanthum! :)
-
i am happy to setup whatever, my question right now is immediately or Tuesday.
...
On 2022-05-20 19:02:13, Roger Dingledine wrote:
Roger Dingledine commented:
bridgestrap needs Tor version 0.4.7.1-alpha or later to run properly (because of the new feature introduced in tpo/core/tor#40209 (closed)).
I guess it used to run a manually-patched-from-source Tor, but having a proper 0.4.7 Tor package installed on the system sounds even better. Whether that comes from Debian backports or deb.torproject.org is fine imo.
-- Antoine Beaupré torproject.org system administration
-
okay, so i just did this:
root@polyanthum:~# apt install tor/bullseye-backports tor-geoipdb/bullseye-backports Reading package lists... Done Building dependency tree... Done Reading state information... Done Selected version '0.4.7.7-1~bpo11+1' (Debian Backports:bullseye-backports [amd64]) for 'tor' Selected version '0.4.7.7-1~bpo11+1' (Debian Backports:bullseye-backports [all]) for 'tor-geoipdb' Suggested packages: mixmaster torbrowser-launcher apparmor-utils nyx Recommended packages: torsocks The following packages will be upgraded: tor tor-geoipdb 2 upgraded, 0 newly installed, 0 to remove and 2 not upgraded. Need to get 3483 kB of archives. After this operation, 628 kB of additional disk space will be used. Get:1 https://mirror.hetzner.de/debian/packages bullseye-backports/main amd64 tor-geoipdb all 0.4.7.7-1~bpo11+1 [1466 kB] Get:2 https://mirror.hetzner.de/debian/packages bullseye-backports/main amd64 tor amd64 0.4.7.7-1~bpo11+1 [2018 kB] Fetched 3483 kB in 0s (16.6 MB/s) [master 8626b214] saving uncommitted changes in /etc prior to apt run 1 file changed, 0 insertions(+), 0 deletions(-) (Reading database ... 67063 files and directories currently installed.) Preparing to unpack .../tor-geoipdb_0.4.7.7-1~bpo11+1_all.deb ... Unpacking tor-geoipdb (0.4.7.7-1~bpo11+1) over (0.4.5.10-1~deb11u1) ... Preparing to unpack .../tor_0.4.7.7-1~bpo11+1_amd64.deb ... Unpacking tor (0.4.7.7-1~bpo11+1) over (0.4.5.10-1~deb11u1) ... Setting up tor (0.4.7.7-1~bpo11+1) ... Setting up tor-geoipdb (0.4.7.7-1~bpo11+1) ... Processing triggers for man-db (2.9.4-2) ... Scanning processes... Scanning candidates... Scanning linux images... Running kernel seems to be up-to-date. Restarting services... systemctl restart user@2150.service Job for user@2150.service failed because the control process exited with error code. See "systemctl status user@2150.service" and "journalctl -xe" for details. No containers need to be restarted. No user sessions are running outdated binaries.Edited by anarcat -
AKA, we're now running the backported tor and tor-geoipdb things on polyanthum. notice needrestart failing to restart
user@2150.service: that's bridgestrap that's still broken, i think. i restarted it by hand, and now it seems to be running:root@polyanthum:~# getent passwd 2150 bridgestrap:x:2150:2150:bridgestrap role account,,,,:/home/bridgestrap:/bin/dash root@polyanthum:~# systemctl status user@2150.service ● user@2150.service - User Manager for UID 2150 Loaded: loaded (/lib/systemd/system/user@.service; static) Active: failed (Result: exit-code) since Sat 2022-05-21 01:14:06 UTC; 1min 54s ago Docs: man:user@.service(5) Process: 306918 ExecStart=/lib/systemd/systemd --user (code=exited, status=219/CGROUP) Main PID: 306918 (code=exited, status=219/CGROUP) CPU: 369us May 21 01:14:06 polyanthum systemd[1]: Starting User Manager for UID 2150... May 21 01:14:06 polyanthum systemd[1]: user@2150.service: Main process exited, code=exited, status=219/CGROUP May 21 01:14:06 polyanthum systemd[1]: user@2150.service: Failed with result 'exit-code'. May 21 01:14:06 polyanthum systemd[1]: Failed to start User Manager for UID 2150. root@polyanthum:~# systemctl restart user@2150.service root@polyanthum:~# systemctl restart user@2150.service root@polyanthum:~# systemctl status user@2150.service ● user@2150.service - User Manager for UID 2150 Loaded: loaded (/lib/systemd/system/user@.service; static) Active: active (running) since Sat 2022-05-21 01:16:09 UTC; 1s ago Docs: man:user@.service(5) Main PID: 307151 (systemd) Status: "Startup finished in 33ms." Tasks: 17 Memory: 24.6M CPU: 513ms CGroup: /user.slice/user-2150.slice/user@2150.service ├─app.slice │ └─bridgestrap.service │ ├─307165 /home/bridgestrap/bin/bridgestrap -addr 127.0.0.1:5001 -cache /home/bridgestrap/bridgestrap-cache.bin -collector-metrics /home/bridgestrap/collector-metrics.log -lo> │ ├─307170 tor -f /tmp/tor-datadir-261126895/torrc-2151028893 │ └─307171 /usr/bin/obfs4proxy -enableLogging -logLevel DEBUG └─init.scope ├─307151 /lib/systemd/systemd --user └─307152 (sd-pam) May 21 01:16:09 polyanthum systemd[307151]: Listening on GnuPG cryptographic agent and passphrase cache (access for web browsers). May 21 01:16:09 polyanthum systemd[307151]: Listening on GnuPG cryptographic agent and passphrase cache (restricted). May 21 01:16:09 polyanthum systemd[307151]: Listening on GnuPG cryptographic agent (ssh-agent emulation). May 21 01:16:09 polyanthum systemd[307151]: Listening on GnuPG cryptographic agent and passphrase cache. May 21 01:16:09 polyanthum systemd[307151]: Reached target Sockets. May 21 01:16:09 polyanthum systemd[307151]: Reached target Basic System. May 21 01:16:09 polyanthum systemd[1]: Started User Manager for UID 2150. May 21 01:16:09 polyanthum systemd[307151]: Started Daemon that exposes an API that allows for the testing of Tor bridges.. May 21 01:16:09 polyanthum systemd[307151]: Reached target Main User Target. May 21 01:16:09 polyanthum systemd[307151]: Startup finished in 33ms.we still need to put the backports stuff in puppet, and i have a branch to work on that in puppet, but that will need to wait for next week (tuesday) as I don't want to do a fleet wide deploy on a friday night, especially since i'm leaving for a long weekend here.
hopefully that will get you folks out of trouble for now!
mentioned in issue tpo/anti-censorship/bridgestrap#34 (closed)
-
I think so too!
I will let meskio take the more serious look on monday and then [also let him] close this ticket. Thanks!
Edited by Roger Dingledine -
before this ticket is closed, I need to fix unattended-upgrades to follow backports for those packages.
...
On 2022-05-21 08:01:55, Roger Dingledine (@arma) wrote:
I will let meskio take the more serious look on monday and then close this ticket. Thanks!
-- Antoine Beaupré torproject.org system administration
mentioned in issue tpo/anti-censorship/bridgestrap#17 (closed)
mentioned in issue #40771 (closed)
marked this issue as related to #40771 (closed)
-
before this ticket is closed, I need to fix unattended-upgrades to follow backports for those packages.
that is now done (in #40771 (closed)), thank you for your patience.
mentioned in issue #40773 (closed)
mentioned in issue #41676 (closed)