Skip to content
Snippets Groups Projects
Commit af7a3698 authored by zen's avatar zen
Browse files

Merge branch 'puppet-merge-costs'

parents 8d682901 6cddfaaa
No related branches found
No related tags found
1 merge request!71tpa-rfc-77: add cost and timeline estimates for the Puppet merge
Pipeline #252229 failed
Stage: build
Stage: test
Hide whitespace changes
Inline Side-by-side
policy/tpa-rfc-77-puppet-merge.md
+ 155
− 46
View file @ af7a3698
...@@ -42,32 +42,36 @@ This phase ensures that, once Tails code is copied to Tor's Puppet Control repo: ...@@ -42,32 +42,36 @@ This phase ensures that, once Tails code is copied to Tor's Puppet Control repo:
### Converge in structure ### Converge in structure
- Tails: Tails:
- Switch from Git submodules to using g10k ([#41974](https://gitlab.torproject.org/tpo/tpa/team/-/issues/41974))
- Remove ENC configuration, Tails don't really use it and the Puppet server switch will implement Tor's instead - (1.1) Switch from Git submodules to using g10k ([#41974](https://gitlab.torproject.org/tpo/tpa/team/-/issues/41974))
- Move node definitions under `manifests/nodes.pp` to roles - (1.2) Remove ENC configuration, Tails don't really use it and the Puppet server switch will implement Tor's instead
- Switch to the directory structure used by Tor: - (1.3) Move node definitions under `manifests/nodes.pp` to roles
- Move custom non-profile modules (`bitcoind`, `borgbackup`, `etckeeper`, `gitolite`, `rbac`, `reprepro`, `rss2email`, `tails`, `tirewall` and `yapgp`) to `legacy/`. Note: there are no naming conflicts in this case. - (1.4) Switch to the directory structure used by Tor:
- Make sure to leave only 3rd party modules under `modules/`. There are 2 naming conflicts here (`unbound` and `network`): Tails uses these from voxpupuli and Tor uses custom ones in `legacy/`, so in these cases we deprecate the Tor ones in favor of voxpupuli's. - Move custom non-profile modules (`bitcoind`, `borgbackup`, `etckeeper`, `gitolite`, `rbac`, `reprepro`, `rss2email`, `tails`, `tirewall` and `yapgp`) to `legacy/`. Note: there are no naming conflicts in this case.
- Rename `hieradata` to `data` - Make sure to leave only 3rd party modules under `modules/`. There are 2 naming conflicts here (`unbound` and `network`): Tails uses these from voxpupuli and Tor uses custom ones in `legacy/`, so in these cases we deprecate the Tor ones in favor of voxpupuli's.
- Rename `profiles` to `site` - Rename `hieradata` to `data`
- Move default configuration to a new `profile::tails` class and include it in all nodes - Rename `profiles` to `site`
- (1.5) Move default configuration to a new `profile::tails` class and include it in all nodes
### Converge in substance ### Converge in substance
- Tails: Tails:
- Rename all profiles from `tails::profile` to `profile::tails`
- Ensure all exported resources' tags are prefixed with tails_ - (1.6) Rename all profiles from `tails::profile` to `profile::tails`
- Upgrade 3rdparty modules to match TPA versions - (1.7) Ensure all exported resources' tags are prefixed with tails_
- Tor: - (1.8) Upgrade 3rdparty modules to match TPA versions
- Install all `3rdparty` modules that are used by Tails but not by Tor
- Isolate all exported resources and collectors using tags Tor:
- Move default configuration to a new `profile::tpa` class and include it in all nodes
- Enforce signed commits - (1.9) Install all `3rdparty` modules that are used by Tails but not by Tor
- Ensure all private data is moved to Trocla and publish the repo ([tpo/tpa/team#29387](https://gitlab.torproject.org/tpo/tpa/team/-/issues/29387)) - (1.10) Isolate all exported resources and collectors using tags
- Import the `tails::profile::puppet::eyaml` profile into TPA's `profile::puppet::server` - (1.11) Move default configuration to a new `profile::tpa` class and include it in all nodes
- Copy the EYAML keys from the Tails to the Tor puppet server, and adapt `hiera.yaml` to use them - (1.12) Enforce signed commits
- Upgrade 3rdparty modules to match Tails versions - (1.13) Ensure all private data is moved to Trocla and publish the repo ([tpo/tpa/team#29387](https://gitlab.torproject.org/tpo/tpa/team/-/issues/29387))
- (1.14) Import the `tails::profile::puppet::eyaml` profile into TPA's `profile::puppet::server`
- (1.15) Copy the EYAML keys from the Tails to the Tor puppet server, and adapt `hiera.yaml` to use them
- (1.16) Upgrade 3rdparty modules to match Tails versions
When we say "upgrade", we don't mean to upgrade to the latest upstream When we say "upgrade", we don't mean to upgrade to the latest upstream
version of a module, but to the latest release that is highest version version of a module, but to the latest release that is highest version
...@@ -83,28 +87,28 @@ considering the Tails codebase is generally tidier. ...@@ -83,28 +87,28 @@ considering the Tails codebase is generally tidier.
This phase moves all nodes from one Puppet server to the other: This phase moves all nodes from one Puppet server to the other:
- Copy `legacy` modules from Tails to Tor - (2.1) Copy code (`legacy` modules and profiles) from Tails to Tor
- Copy profiles from Tails to Tor - (2.2) Create a flag that determines whether a node is Tails or TPA and which
- Create a flag that determines whether a node is Tails or TPA and which base class it should include base class it should include, and assign nodes to their corresponding base
- Assign nodes to their corresponding base class using the flag above class using the flag above
- Point Tails nodes to the Tor Puppet server - (2.3) Point Tails nodes to the Tor Puppet server
- Retire the Tails' Puppet server - (2.4) Retire the Tails' Puppet server
## Phase 3: Towards a more homogeneous codebase ## Phase 3: Codebase homogeneity
This phase paves the way towards a cleaner future: This phase paves the way towards a cleaner future:
- Remove all `tails::profile::puppet` profiles - (3.1) Remove all `tails::profile::puppet` profiles
- Merge the 8 conflicting Tails and TPA profiles: - (3.2) Merge the 8 conflicting Tails and TPA profiles:
- `grub` - `grub`
- `limesurvey` - `limesurvey`
- `mta` - `mta`
- `nginx` - `nginx`
- `podman` - `podman`
- `rspamd` - `rspamd`
- `sudo` - `sudo`
- `sysctl` - `sysctl`
- Move the remaining 114 non-conflicting Tails profiles to `profile` (without `::tails`) - (3.3) Move the remaining 114 non-conflicting Tails profiles to `profile` (without `::tails`)
At this point, we'll have 244 profiles. At this point, we'll have 244 profiles.
...@@ -127,14 +131,119 @@ services on a need-to basis, and progress in the merge roadmap. ...@@ -127,14 +131,119 @@ services on a need-to basis, and progress in the merge roadmap.
# Costs # Costs
TODO: try to break down costs of each of the above steps, to get a To estimate costs of tasks in days of work, We use the same parameters as
broad idea of how long all this work is going to take (and generate proposed in [Jacob Kaplan-Moss' estimation
the below timeline). technique](https://jacobian.org/2021/may/25/my-estimation-technique/).
"Complexity" estimates the size of a task in days, accounting for all other
things a worker has to deal with during a normal workday:
| Complexity | Time |
| --- | --- |
| small | 1 day |
| medium | 3 days |
| large | 1 week (5 days) |
| extra-large | 2 weeks (10 days) |
"Uncertainty" is a scale factor applied to the length to get a pessimistic
estimate if things go wrong:
| Uncertainty Level | Multiplier |
| --- | --- |
| low | 1.1 |
| moderate | 1.5 |
| high | 2.0 |
| extreme | 5.0 |
## Per-task worst-case duration estimate
| Task | Codebase | Complexity | Uncertainty | Expected (days) | Worst case (days) |
| --- | --- | --- | --- | --- | --- |
| (1.1) Switch to g10k | Tails | small | high | 2 | 4 |
| (1.2) Remove ENC | Tails | small | low | 1 | 1.1 |
| (1.3) Move nodes do roles | Tails | medium | low | 3 | 3.3 |
| (1.4) Switch directory structure | Tails | small | moderate | 1 | 1.5 |
| (1.5) Create default profile | Tails | small | moderate | 1 | 1.5 |
| (1.6) Rename Tails profiles | Tails | small | low | 1 | 1.1 |
| (1.7) Prefix exported resources | Tails | medium | low | 3 | 3.3 |
| (1.8) Upgrade 3rd party modules | Tails | large | moderate | 5 | 7.5 |
| (1.9) Install missing 3rd party modules | Tor | small | low | 1 | 1.1 |
| (1.10) Prefix exported resources | Tor | medium | low | 3 | 3.3 |
| (1.11) Create default profile | Tor | small | moderate | 1 | 1.5 |
| (1.12) Enforce signed commits | Tor | medium | moderate | 3 | 4.5 |
| (1.13) Move private data to Trocla | Tor | large | moderate | 5 | 7.5 |
| (1.14) Publish repository | Tor | large | moderate | 5 | 7.5 |
| (1.15) Enable EYAML | Tor | small | low | 1 | 1.1 |
| (1.16) Upgrade 3rd party modules | Tor | x-large | high | 10 | 20 |
| (2.1) Copy code | Tor | small | low | 1 | 1.1 |
| (2.2) Differentiate Tails and Tor nodes | Tor | small | moderate | 1 | 1.5 |
| (2.3) Switch Tails' nodes to Tor's Puppet server | Tor | large | extreme | 5 | 25 |
| (2.4) Retire the Tails Puppet server | Tor | small | low | 1 | 1.1 |
| (3.1) Ditch the Tails' Puppet profile | Tor | small | low | 1 | 1.1 |
| (3.2) Merge conflicting profiles | Tor | large | extreme | 5 | 25 |
| (3.3) Ditch the `profile::tails` namespace | Tor | small | low | 1 | 1.1 |
## Per-phase worst-case time estimate
| Task | Worst case (days) | Worst case (weeks) |
| --- | --- | --- |
| Phase 1: Codebase preparation | 69.8 | 17.45 |
| Phase 2: Puppet server switch | 28.7 | 7.2 |
| Phase 3: Codebase homogeneity | 27.2 | 6.8 |
Worst case duration: 125.7 days =~ 31.5 weeks
# Timeline # Timeline
TODO: detail when we are planning each phase, broadly (say, per The following parallel activities will probably influence (i.e. delay) this
quarter or month) plan:
- Upgrade to Debian Trixie: maybe start on March, ideally finish by the end of
2025
- North-hemisphere summer vacations
Base on the above estimates, taking into account the potential delays, and
stretching it a bit for a worst case scenario, here is a rough per-month
timeline:
- March (all Tails):
- (1.1) Switch to g10k
- (1.2) Remove ENC
- (1.3) Move nodes to roles
- (1.4) Switch directory structure
- April:
- (1.5) Create default profile
- (1.6) Rename Tails profiles
- (1.7) Prefix exported resources
- (1.8) Upgrade 3rd party modules (Tails)
- May:
- (1.8) Upgrade 3rd party modules (Tails) (continuation)
- (1.9) Install missing 3rd party modules (Tor)
- (1.10) Prefix exported resources (Tor)
- (1.11) Create default profile (Tor)
- June (all Tor from now on):
- (1.12) Enforce signed commits
- (1.13) Move private data to Trocla
- July:
- (1.14) Publish repository
- (1.15) Enable EYAML
- (1.16) Upgrade 3rd party modules
- August:
- (1.16) Upgrade 3rd party modules (continuation)
- September:
- (2.1) Copy code
- (2.2) Differentiate Tails and Tor nodes
- (2.3) Switch Tails' nodes to Tor's Puppet server
- October:
- (2.3) Switch Tails' nodes to Tor's Puppet server (continuation)
- November:
- (2.4) Retire the Tails Puppet server
- (3.1) Ditch the Tails' Puppet profile
- Devember:
- (3.2) Merge conflicting profiles
- January:
- (3.2) Merge conflicting profiles (continuation)
- (3.3) Ditch the `profile::tails` namespace
# Alternatives considered # Alternatives considered
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment