TPA team issueshttps://gitlab.torproject.org/tpo/tpa/team/-/issues2021-02-08T19:28:20Zhttps://gitlab.torproject.org/tpo/tpa/team/-/issues/40161Delete branches from snowflake-webext2021-02-08T19:28:20ZArlo BreaultDelete branches from snowflake-webextPlease remove these stale branches,
```
remotes/origin/14-update-bug-reporting-links-for-gitlab
remotes/origin/15-sync-translation-messages-json-with-static-index-html
```Please remove these stale branches,
```
remotes/origin/14-update-bug-reporting-links-for-gitlab
remotes/origin/15-sync-translation-messages-json-with-static-index-html
```anarcatanarcathttps://gitlab.torproject.org/tpo/tpa/team/-/issues/40167TPA-RFC-10: adopt jenkins retirement plan2022-03-28T19:15:32ZanarcatTPA-RFC-10: adopt jenkins retirement planI've been brainstorming here and there about how we could eventually replace Jenkins. Now that people are getting excited about GitLab CI and that service is taking up more and more of our time, it seems like a good time to start plannin...I've been brainstorming here and there about how we could eventually replace Jenkins. Now that people are getting excited about GitLab CI and that service is taking up more and more of our time, it seems like a good time to start planning for Jenkins retirement.
Requires jenkins service docs first (#40166).
update: followup of this proposal is in https://gitlab.torproject.org/groups/tpo/-/milestones/27Retire Jenkinsanarcatanarcat2021-04-15https://gitlab.torproject.org/tpo/tpa/team/-/issues/40169Torbutton's post-receive should be updated?2021-02-10T15:15:01ZMatthew FinkelTorbutton's post-receive should be updated?I think torbutton's hook should be updated. I don't know what it contains, but I'm guessing there is something related to Trac. We're also pushing to dip.tpo, so that could be updated, too.
```
remote: To dip.torproject.org:tpo/applicat...I think torbutton's hook should be updated. I don't know what it contains, but I'm guessing there is something related to Trac. We're also pushing to dip.tpo, so that could be updated, too.
```
remote: To dip.torproject.org:tpo/applications/torbutton
remote: 28abe803..edad4c65 master -> master
remote: == irc-message ==
remote: == per-repo-hook ==
remote: run-parts: executing /srv/git.torproject.org/git-helpers/post-receive-per-repo.d/torbutton/trigger-trac torbutton /tmp/tmp.Vxh98EaAiT
remote: [/srv/git.torproject.org/git-helpers/post-receive-per-repo.d/torbutton/trigger-trac] Triggering trac update for torbutton on troodi.torproject.org.
remote: ssh: Could not resolve hostname troodi.torproject.org: Name or service not known
remote: run-parts: /srv/git.torproject.org/git-helpers/post-receive-per-repo.d/torbutton/trigger-trac exited with return code 255
remote: == xx-jenkins-trigger ==
remote: [hook[22435]] Triggering jenkins build for (https://git.torproject.org/torbutton.git, master, edad4c65724fecfcacec5afd7e8c394f850d9f1d).
remote: No git jobs using repository: https://git.torproject.org/torbutton.git and branches: master
remote: No Git consumers using SCM API plugin for: https://git.torproject.org/torbutton.git
remote: [hook[22435]] Jenkins triggers done.
To ssh://git-rw.torproject.org/torbutton.git
28abe803..edad4c65 master -> master
```
(Probably CC @ahf or @hiro)anarcatanarcathttps://gitlab.torproject.org/tpo/tpa/team/-/issues/40188make sure SNI can be disabled on fastly2021-03-29T15:26:50Zanarcatmake sure SNI can be disabled on fastlyFastly is [migrating towards SNI for their HTTPS connections](https://docs.fastly.com/changes/significant/2020/03/09/special), and have found that we use non-SNI connections for some things. We have done a round of checks to see what act...Fastly is [migrating towards SNI for their HTTPS connections](https://docs.fastly.com/changes/significant/2020/03/09/special), and have found that we use non-SNI connections for some things. We have done a round of checks to see what actually does that, but now the deadlines are approaching, and we need to make sure everything is in order.
The deadlines are:
> - **March 15, 2021** - The Shared and Procured section under your account will become read only.
> - **March 22-25, 2021** - Domains on shared and shared wildcard certificates will be automatically migrated to Fastly TLS.
Affected?
* TBB: @sysrqb, @gk
* bwauth: @juga
This is a followup to #40181 and an email discussion thread that Fastly started on January 31st (`Message-Id: CAEYXjdZK5Fy05c9viWZHCw8HozCfX8bkrJQ5q9zOq1G62K7r0Q@mail.gmail.com`).anarcatanarcathttps://gitlab.torproject.org/tpo/tpa/team/-/issues/40200gettor jenkins builds the wrong set of builds, along with the right ones2021-04-09T12:34:06Zemmapeelgettor jenkins builds the wrong set of builds, along with the right onesI am setting up the translations for the GetTor website, and I think I have found and issue, but I am not sure if this is the problem:
The lektors with localization setup should trigger the job 'project' 'lektor-website-translation', de...I am setting up the translations for the GetTor website, and I think I have found and issue, but I am not sure if this is the problem:
The lektors with localization setup should trigger the job 'project' 'lektor-website-translation', defined here: https://gitweb.torproject.org/project/jenkins/jobs.git/tree/lektor-website.yaml#n328 and gettor is on that list.
But it seems that when I push to the repo, I also trigger the job project 'lektor website': https://gitweb.torproject.org/project/jenkins/jobs.git/tree/lektor-website.yaml#n310 (without translation).
In jenkins I can see jobs like: https://jenkins.torproject.org/job/lektor-website-gettor-any/ that are not supposed to be there, although I can also see the right jobs, as https://jenkins.torproject.org/job/lektor-website-gettor-translation-install/
Let me know if I can be of any help testing.weasel (Peter Palfrader)weasel (Peter Palfrader)https://gitlab.torproject.org/tpo/tpa/team/-/issues/40210Modify redirect for onionperf.torproject.org2021-04-08T20:07:13ZirlModify redirect for onionperf.torproject.orgNew URL should be: https://gitlab.torproject.org/tpo/metrics/team/-/wikis/onionperf
Context: https://gitlab.torproject.org/tpo/metrics/team/-/issues/4New URL should be: https://gitlab.torproject.org/tpo/metrics/team/-/wikis/onionperf
Context: https://gitlab.torproject.org/tpo/metrics/team/-/issues/4anarcatanarcathttps://gitlab.torproject.org/tpo/tpa/team/-/issues/40246write access to anti censorship repos2021-05-12T19:46:14Zmeskiomeskio@torproject.orgwrite access to anti censorship reposI will need write access to the anti censorship repos in git-rw.torproject.org.
Not sure if I should get access to all of them at once or just the ones I'm going to touch right now (bridgedb and rdsys).I will need write access to the anti censorship repos in git-rw.torproject.org.
Not sure if I should get access to all of them at once or just the ones I'm going to touch right now (bridgedb and rdsys).anarcatanarcathttps://gitlab.torproject.org/tpo/tpa/team/-/issues/40247Change default anti-censorship branches on git.torproject.org2021-06-08T06:47:20ZCecylia BocovichChange default anti-censorship branches on git.torproject.orgI've been working on https://gitlab.torproject.org/tpo/anti-censorship/team/-/issues/6 and have now managed to make all of the changes that I have permissions to make. What remains are to 1) push new branches to the git.torproject.org re...I've been working on https://gitlab.torproject.org/tpo/anti-censorship/team/-/issues/6 and have now managed to make all of the changes that I have permissions to make. What remains are to 1) push new branches to the git.torproject.org repositories I don't have write access to, and 2) make main the default branch when cloning from git.torproject.org
For the first item, I don't have and would like write access to the following repositories:
- [gettor-web](https://gitweb.torproject.org/project/web/gettor-web.git/)
- [bridgedb-admin](https://gitweb.torproject.org/project/bridges/bridgedb-admin.git/)
- [goptlib](https://gitweb.torproject.org/pluggable-transports/goptlib.git/)
- [meek](https://gitweb.torproject.org/pluggable-transports/meek.git/)
- [obfs4](https://gitweb.torproject.org/pluggable-transports/obfs4.git/)
For the second item, I'm not sure how to accomplish this. This was the output from your script that I ran:
```
INFO: SSH remote detected, trying to fix default branch with: ('ssh', 'git@git-rw.torproject.org', 'git symbolic-ref HEAD main')
bad command: git symbolic-ref HEAD main
WARNING: failed to change HEAD on remote with SSH
INFO: trying to delete old branch master from remote
remote: warning: deleting the current branch
remote: + refs/heads/master admin/services/gettor cohosh DENIED by fallthru
remote: error: hook declined to update refs/heads/master
To git-rw.torproject.org:admin/services/gettor.git
! [remote rejected] master (hook declined)
error: failed to push some refs to 'git-rw.torproject.org:admin/services/gettor.git'
```
So I think it's the `WARNING: failed to change HEAD on remote with SSH` that failed? If possible I would like to avoid deleting the master branch, and just make it so that when a user clones the repo, the main branch is checked out by default. Is it possible?anarcatanarcathttps://gitlab.torproject.org/tpo/tpa/team/-/issues/40250Create dusk@torproject.org alias to giving@2021-05-12T18:11:41ZGeorge KadianakisCreate dusk@torproject.org alias to giving@Hello! It would be great if you could create dusk@torproject.org and make it an alias of giving@torproject.org .
Thanks a lot!Hello! It would be great if you could create dusk@torproject.org and make it an alias of giving@torproject.org .
Thanks a lot!anarcatanarcathttps://gitlab.torproject.org/tpo/tpa/team/-/issues/40253Give meskio access to polyanthum (BridgeDB)2021-05-18T16:08:37ZCecylia BocovichGive meskio access to polyanthum (BridgeDB)```
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Please provide meskio with access to polyanthum and add him to the bridgedb group.
-----BEGIN PGP SIGNATURE-----
iQIzBAEBCAAdFiEEWmGM6ECIOUK68TNPAJ3jef2be5AFAmCelJcACgkQAJ3jef2b
e5CLS...```
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Please provide meskio with access to polyanthum and add him to the bridgedb group.
-----BEGIN PGP SIGNATURE-----
iQIzBAEBCAAdFiEEWmGM6ECIOUK68TNPAJ3jef2be5AFAmCelJcACgkQAJ3jef2b
e5CLSQ/9HyjlfTbAVqghpEmqZqpLgpUvaQC0k+9Qs5P+01bM4PNWC4ynras4ve63
ZzTXZ/qpyIAqHGVwQz3Fn4W6PdbBWXl1RT2eLVZH4TvYP2+AJxXjr6eH+IJZa/3w
+Aw7fPsWysW0b3nwcygrkTsNwbHFuqrzJ4PVH71kWaRzK2L4gCpAa2RSnVur+oqA
kaQKNOmVUDbRYyh5PY4njN5wQ2GgJQ9AlAgCaJvZeymXxwYVCn8zr2aHvRBShUEC
qooDyi8kLltyXkWJcFk07ss/0Yo00R7DwFDX69UpvpzwXSMd/4BoZiNOhDT3aebN
rL+3gtKl6EGK86bGqbRWbDtmKAjmwpRgZrthtd5BWAeZvJMh7Epvm+AJVXBZFJCD
6D90fBaPflRNQ+XEkDdToOtwZ4/t7Ej7IQ0c4EAVZzcslo8G6EGYgk1EieElk0x0
Gidpu8iZ29DuEznjbD13w4fTykaP2Pof5rv71fo1CHf/gDLQT8CbxWfo4PxlPzMw
rYVPSBOoc5rqgSazI03PXUmaPrm1HlnAtcGqTabFnXjVbHViBP+2Cql2qnC3tQSg
anOVg2ZLEQpsX3SHY+7/NRmX7kOPM78i1icDOF2OloT9oDJDuPo2/ecHupCBedWq
XDswzZ6L3lPd3jvtgBwROlVOMMvMDEBpt0Zw439bjXoxvFOjWdg=
=km8W
-----END PGP SIGNATURE-----
```anarcatanarcathttps://gitlab.torproject.org/tpo/tpa/team/-/issues/40273Please refresh boklm's gpg key in ldap2021-05-31T15:00:37ZboklmPlease refresh boklm's gpg key in ldapI added new subkeys to my key [3E39CEABFC69F6F7](/uploads/fccf6ad54f81776a8b8af1e61f3b653a/3E39CEABFC69F6F7-pubkey-20210531.asc).I added new subkeys to my key [3E39CEABFC69F6F7](/uploads/fccf6ad54f81776a8b8af1e61f3b653a/3E39CEABFC69F6F7-pubkey-20210531.asc).anarcatanarcathttps://gitlab.torproject.org/tpo/tpa/team/-/issues/40274Please monitor HTTP response time for https://metrics.torproject.org/2021-06-19T08:56:51ZirlPlease monitor HTTP response time for https://metrics.torproject.org/Anything over 10 seconds should raise an alarm, anything other than 200 should raise an alarm.Anything over 10 seconds should raise an alarm, anything other than 200 should raise an alarm.irlirlhttps://gitlab.torproject.org/tpo/tpa/team/-/issues/40280Please install python3-prometheus-client on colchifolium and corsicum2021-06-09T23:15:54ZirlPlease install python3-prometheus-client on colchifolium and corsicumanarcatanarcathttps://gitlab.torproject.org/tpo/tpa/team/-/issues/40294Prepare for Web Hosting on arti.torproject.org2023-08-28T15:41:49ZAlexander Færøyahf@torproject.orgPrepare for Web Hosting on arti.torproject.orgHello!
As part of the big transition towards Arti, we are going to create a place where current and hopefully upcoming contributors can find news and information about the Arti development project efforts. The network team and the comms...Hello!
As part of the big transition towards Arti, we are going to create a place where current and hopefully upcoming contributors can find news and information about the Arti development project efforts. The network team and the comms team would like to have a place where we can host entirely static web content about the Arti project.
I know we cannot do Gitlab CI pages with custom domains yet, so we may have to live with a slightly more complicated sync setup at first, but *ideally* at some point it would be awesome if it can be run via Gitlab CI and Gitlab Pages :-)
Is this something we can do? We don't have the page yet, this ticket is just to prepare for it and start the discussion.2021-07-01https://gitlab.torproject.org/tpo/tpa/team/-/issues/40304Can you check the syslog on polyanthum for me?2021-06-16T18:36:57ZCecylia BocovichCan you check the syslog on polyanthum for me?Tor's PT process keeps failing unexpectedly and I'm starting to suspect it might be the OOM killer. Can you take a quick look in /var/log/syslog? (I don't have permissions for that)Tor's PT process keeps failing unexpectedly and I'm starting to suspect it might be the OOM killer. Can you take a quick look in /var/log/syslog? (I don't have permissions for that)anarcatanarcathttps://gitlab.torproject.org/tpo/tpa/team/-/issues/40305Having trouble signing into grafana22021-06-16T21:16:48ZCecylia BocovichHaving trouble signing into grafana2I can no longer login to view the dashboards at https://grafana2.torproject.org
I've been using the username tor-metrics and a password that phw set up when we first started using it for bridgestrap metrics. Did this get reset? I know ...I can no longer login to view the dashboards at https://grafana2.torproject.org
I've been using the username tor-metrics and a password that phw set up when we first started using it for bridgestrap metrics. Did this get reset? I know there were some temporary accounts added for the hackweek. Maybe it got deleted along with those?anarcatanarcathttps://gitlab.torproject.org/tpo/tpa/team/-/issues/40340gitlab runner ci-runner-x86-03-shadow: higher artifact size limit2021-10-21T18:40:36ZJim Newsomegitlab runner ci-runner-x86-03-shadow: higher artifact size limitTrying to run some larger simulations now, and hitting the artifact size limit after completion: https://gitlab.torproject.org/jnewsome/sponsor-61-sims/-/jobs/28578#L9511
In that sim I'd increased the tor network size from 1% to 10%, bu...Trying to run some larger simulations now, and hitting the artifact size limit after completion: https://gitlab.torproject.org/jnewsome/sponsor-61-sims/-/jobs/28578#L9511
In that sim I'd increased the tor network size from 1% to 10%, but was still only simulating 10 minutes (vs the ~60m we'll probably want).
IIRC pastly previously recommended ~100 GB of storage for simulations; maybe try that?Jérôme Charaouilavamind@torproject.orgJérôme Charaouilavamind@torproject.orghttps://gitlab.torproject.org/tpo/tpa/team/-/issues/40364implement mechanism to deploy for static components from GitLab CI instead of...2021-10-21T15:53:22Zanarcatimplement mechanism to deploy for static components from GitLab CI instead of Jenkinsas part of jenkin's retirement (#40218), a key component is to enable GitLab CI to publish content into the static mirror system.
This is the [critical website build](https://gitlab.torproject.org/tpo/tpa/team/-/wikis/policy/tpa-rfc-10-...as part of jenkin's retirement (#40218), a key component is to enable GitLab CI to publish content into the static mirror system.
This is the [critical website build](https://gitlab.torproject.org/tpo/tpa/team/-/wikis/policy/tpa-rfc-10-jenkins-retirement#critical-website-builds) of the [TPA-RF-10 jenkins retirement](https://gitlab.torproject.org/tpo/tpa/team/-/wikis/policy/tpa-rfc-10-jenkins-retirement#critical-website-builds) proposal. In the proposal, we stated this:
> Critical websites should be built by GitLab CI just like non-critical sites, but must be pushed to the static mirror system somehow. The GitLab Pages data source (currently the main GitLab server) should be used as a "static source" which would get triggered by a GitLab web hook after a successful job.
>
> The receiving end of that web hook would be a new service, also running on the GitLab Pages data source, which would receive hook notifications and trigger the relevant static component updates to rsync the files to the static mirror system.
>
> As an exception to the "users migrate their own jobs" rule, TPA and the web team will jointly oversee the implementation of the integration between GitLab CI and the static mirror system. Considering the complexity of both systems, it is unlikely the web team or TPA will be in a position to individually implement this solution.
In the Jenkins retirement ticket (#40218) we expanded on this and @lavamind suggested we use the [webhook daemon](https://packages.debian.org/buster/webhook) to:
> [deploy things directly from GitLab CI](https://about.gitlab.com/blog/2021/02/05/ci-deployment-and-environments/). So actually, we'd have these two options to choose from.
>
> The first:
>
> Webhook is a lightweight configurable tool written in Go, that allows you to easily create HTTP endpoints (hooks) on your server, which you can use to execute configured commands. You can also pass data from the HTTP request (such as headers, payload or query variables) to your commands. webhook also allows you to specify rules which have to be satisfied in order for the hook to be triggered.
>
> The general idea is that we'd configure a Webhook on each GitLab repository with a secret in the `X-GitLab-Token` header to fire when a job succeeds and whose endpoint would be a `webhook` daemon running on the static master. That daemon's job would be to validate the header token and on success launch a predetermined command-line with one or more arguments pulled in from the webhook payload.
>
> The second:
>
> We can also decide to go all-in with GitLab CI and set up a `deploy:` job that will push the build artifacts from CI directly to the server directly and trigger `static-update-component`. To pull this off we'd have to put an SSH private key in GitLab (as a protected variable), and set up `authorized_keys` on the static master to permit the necessary commands only.
The workflow would be:
1. user pushes a change to git, which ...
1. triggers a CI pipeline
1. CI runner picks up the jobs and builds the website, pushes the artifacts back to GitLab
1. GitLab fires a webhook
1. webhook receives the ping and authenticates against a configuration, mapping to a given static-sync component
1. hopefully the webhook has details of the artifacts in-band, otherwise it needs to fetch the path to the artifacts here)
1. webhook downloads the artifact into the static component source path
1. webhook fires the static-update-component command on the right project
A few concerns I have with this approach:
* we need a configuration file containing the GitLab webhook tokens
* this is the webhook configuration file
* we need to figure out how to map webhook pings to components, maybe by adding a variable to the CI configuration?
* this is *also* the webhook configuration, the secret webhook tokens maps to one and only one site URL, which then maps correctly into the static component
* this does mean that a new static component like this needs an entry in two places: the old `static-components.yaml` file, and Hiera
* we need to keep the webhook tokens in sync
* this is a one-shot thing
* the webhook software needs to be configured to fire up *some* command, which command probably needs to parse the above configuration file, which, maybe, means we could just write a whole new daemon instead of reusing webhook?
* we keep the webhook simple and move most business logic into a one-shot script, with the hope that not running a custom-made daemon will simplify the design
* all the above configuration is a manual snowflake just for us, which adds to an already very complicated static-sync design
* that will be the case until we get rid of the static-component system. and hopefully it will be slightly simpler (and better documented) than the Jenkins setup
In any case, we need to implement this, soonish. We have a spot check coming up in september for the retirement, and there has been little progress on the website front, so we need to start doing something here. It was agreed with @lavamind that we could use the static blog website as a test for this deployment, but we could also use status.torproject.org as well.
I favor the webhook + artifacts approach instead of the "SSH deployment" approach. My rationale is:
> I have typically been uncomfortable with giving GitLab SSH access on servers, and particularly on the static mirror infrastructure, so we have not picked that approach so far. But maybe it should be reconsidered, especially with hardened `authorized_keys` files?
>
> Might certainly be simpler than writing a webook, but it would still require some configuration on both ends, and all mostly manual, which is kind of a drag...
Launch checklist:
- [x] figure out how to configure webhook and how to integrate (if at all?) with the static component configuration
- [x] decide where to deploy the webhook compnent (new static source)
- [x] name things ("puller", "notifier", "static source", the entire project needs a better name), names are:
* static source hostname: gitlab-shim (we previously used static-gitlab-shim-source, but that broke bacula, see tpo/tpa/team#40416)
* puller: static-gitlab-shim-pull
* notifier: "the webhook" should be enough
* class name: `staticsync::gitlab_shim`
- [x] (re-)create a VM for the static source
- [x] write the puller and deploy status.torproject.org locally
- [x] write the notifier (it's the `webhook` package with a simple config managed by puppet0
- [x] add the new VM as a `staticsync::static_source` in puppet
- [x] sudo configuration
- [x] deployment: replace the current status.torproject.org source with the new source (involves changing `static-components.yaml`) (https://gitlab.torproject.org/tpo/tpa/status-site/-/issues/14)
- [x] at this point, status.torproject.org is managed by GitLab CI, celebrate!
- [x] fix webhook timeout issues (e.g. [change timeout](https://docs.gitlab.com/ee/user/project/integrations/webhooks.html#webhook-fails-or-multiple-webhook-requests-are-triggered), daemonize puller, or switch to ssh deployment)
- [x] retire the webhook code
- [x] send an announcement about the research.tpo migration
- [x] create a [gitlab CI template](https://docs.gitlab.com/ee/ci/yaml/index.html#includefile) for the deployment script (see the [template repo](https://gitlab.torproject.org/tpo/tpa/ci-templates) and the [deploy template](https://gitlab.torproject.org/tpo/tpa/ci-templates/-/blob/main/static-shim-deploy.yml))
- [x] migrate another existing hugo site (research.tpo, tpo/web/research#40005)
- [x] document how admins setup such sites
- [x] ~~migrate one lektor site~~
- [x] document how *users* setup such sites (see [the static shim docs](https://gitlab.torproject.org/tpo/tpa/team/-/wikis/service/static-shim#deploying-a-static-site-from-gitlab-ci))
- [x] send an announcement about the upcoming migration and documentation
- [x] make tickets for the individual websites to migrate, part of the roadmap
- [x] finish design docs (clear all TODO items in the `service/static-shim.md` wiki page)
- [x] ~~deploy the test blog with the webhook system~~
- [x] ~~consider standardising the lektor CI pipeline (tpo/web/team#4)~~
- [x] ~~migrate another existing lektor site~~
- [x] ~~migrate *all* hugo sites (`hugo.yaml` in jenkins)~~
- [x] ~~migrate *all* lektor sites (`lektor.yaml` in jenkins)~~
- [x] ~~migrate www.torproject.org (`website.yaml` in jenkins)~~
When this checklist is done:
* a solid procedure for migrating sites is implemented
* a few sites have been migrated as tests
* ~~all the jobs are retired from jenkins~~
* ~~all the repositories are migrated to GitLab~~
* everything is clearly documentedRetire Jenkinsanarcatanarcathttps://gitlab.torproject.org/tpo/tpa/team/-/issues/40367revert back to usrmerge in installers2022-03-16T02:34:32Zanarcatrevert back to usrmerge in installerswe need to revert the changes we did in our installers to do `--no-merged-usr` everywhere (see #34115). the rationale is that the Debian Technical Committee decided that Debian 11 bullseye would be the last release to support a *non*-mer...we need to revert the changes we did in our installers to do `--no-merged-usr` everywhere (see #34115). the rationale is that the Debian Technical Committee decided that Debian 11 bullseye would be the last release to support a *non*-merged-usr configuration, and we should therefore avoid creating new such configurations.
Plus, I actually found problems with the `--no-merged-usr` configuration, as detailed in #34115:
> one impact of mergedusr is actually when it's *not* enabled. for example, with mergedusr, `/usr/sbin/ip` is a valid path, but without, it isn't. so someone (mistakenly, perhaps) hardcoding said path will *fail* on a non-merged system (including any stretch system) while working on a merged system.
So, in other words, we have found problems *without* a merged usr an *no* problem *with* it.
We will also, eventually, need to convert existing installs before the Debian 12 "bookworm" upgrade, but that can happen later. For now, let's just fix the installers.
in #34115, I said:
> worst case, we at least have knobs on how to switch that off everywhere as well. just grep for `--no-merged-usr`.
... so that should be easy enough.anarcatanarcathttps://gitlab.torproject.org/tpo/tpa/team/-/issues/40377chi-node-05 has hardware (memory, disk) issues2022-04-25T18:38:37Zanarcatchi-node-05 has hardware (memory, disk) issueswhen trying to install chi-node-05 (#40365), i had many problems. but most worrying is what seems to be a hardware issue.
the iDRAC says:
```
Persistent correctable memory errors detected on a memory device at location DIMM_A2.
```
I ...when trying to install chi-node-05 (#40365), i had many problems. but most worrying is what seems to be a hardware issue.
the iDRAC says:
```
Persistent correctable memory errors detected on a memory device at location DIMM_A2.
```
I believe DIMM_A5 also has issues based on BIOS warnings at boot.
In general, the console is barely useable and i am not sure the machine will be reliable in any way. need to talk with cymru about this.
Update: there's also warnings from smartd...
```
From: root <root@chi-node-05.torproject.org>
Subject: SMART error (ErrorCount) detected on host: chi-node-05
To: root@chi-node-05.torproject.org
Date: Thu, 09 Sep 2021 19:51:59 +0000
This message was generated by the smartd daemon running on:
host name: chi-node-05
DNS domain: torproject.org
The following warning/error was logged by the smartd daemon:
Device: /dev/bus/0 [megaraid_disk_00] [SAT], ATA error count increased from 0 to 2
Device info:
ST9500620NS, S/N:9XF0AC2F, WWN:5-000c50-0356dad0e, FW:AA02, 500 GB
For details see host's SYSLOG.
You can also use the smartctl utility for further investigation.
Another message will be sent in 24 hours if the problem persists.
```anarcatanarcat