[[TPA-RFC-73|/tpo/tpa/team/-/wikis/policy/tpa-rfc-73-tails-infra-merge-roadmap]] identified Puppet as a bottleneck for the infra merge, as it blocks keeping, migrating and merging several other services. Merging codebases and ditching one of the Puppet servers is a complex move, so in this document we detail how that will be done.
# Proposal
## Goals
### Must have
- One Puppet Server to rule them all
- Adoption of TPA's solution for handling Puppet modules and ENC
- Convergence in Puppet modules versions
- Commit signing (as it's fundamental for Tails' current backup solution)
### Non-goals
This proposal is **not** about:
- completely refactoring and deduplicating code, as that will be done step-by-step while we handle each services individually after the Puppet Server merge
- ditching one way to store secrets in favor of another, as that will be done separately in the future, after both teams had the chance to experience Trocla and hiera-eyaml
- tackling individual service merges, such as backups, dns, monitoring and firewall; these will be tackled individually once all infra is under one Puppet Server
- applying new code standards everywhere; at most, we'll come up with general guidelines that could (maybe should) be used for new code and, in the future, for refactoring
## Phase 1: Codebase preparation
This phase ensures that, once Tails code is copied to Tor's Puppet Control repo:
- Code structure will match and be coherent
- Tails code will not affect Tor's infra and Tor's code will not affect Tails infra
**Note:** Make sure to freeze all Puppet code refactoring on both sides before starting.
### Converge in structure
- Tails:
- switch from Git submodules to using g10k in a monorepo
- remove ENC configuration, Tails don't really use it and the Puppet server switch will implement Tor's instead
- move node definitions under `manifests/nodes.pp` to roles
- switch to the directory structure used by Tor:
- move custom non-profile modules to `legacy/` (monitoring, apache, passenger), leave only 3rd party modules under `modules/`
-`hieradata` → `data`
-`profiles` → `site`
### Converge in substance
- Tails:
- refactor the legacy apache and passenger modules out of existence
- rename all profiles from tails::profile to profile::tails
- ensure all exported resources' tags are prepended with tails_
- Tor:
- install all 3rdparty modules that are used by Tails but not by Tor
- isolate all exported resources and collectors by using tags
- ensure there is a parameter to disable all 'base' functionality (i.e., nothing gets installed on a puppet node that is not explicitly included in the role)
- enforce signed commits
- ensure all private data is moved to trocla and publish the repo
- install eyaml, copy the eyaml keys from the Tails to the Tor puppet server, and adapt hiera.yaml to use them
- Tor and Tails:
- upgrade 3rdparty modules to match versions
## Phase 2: Puppet server switch
This phase moves all nodes from one Puppet server to the other:
- copy homebrew/legacy modules from Tails to Tor
- copy roles and profiles from Tails to Tor
- assign nodes to roles using the ENC
- point Tails nodes to the Tor puppetserver
- retire the Tails' Puppet server
## Phase 3: Towards a more homogeneous codebase
This phase paves the way towards a cleaner future:
- one by one, for each profile in profile::tails
- move the profile to profile::, or
- merge the profile with an existing one in profile::
-[Migrate services to TPA before moving Puppet][]: some of the Tails services heavily depend on others and/or on the network setup (eg. Jenkins Agents on different machines talk to a Jenkins Orchestrator and a Gitolite server hosted on different VMs, then build nightly ISOs that are copied to the web VM and published over HTTP), and migrating these over to TPA's infra would be much more complex than just merging Puppet.
[Migrate services to TPA before moving Puppet]:https://gitlab.torproject.org/tpo/tpa/team/-/wikis/meeting/2024-11-11/#per-service-notes
# References
-[TPA-RFC-73: Tails infra merge roadmap - Puppet Server][]
-[Merge Tails' and Tor's Puppet servers (tpo/tpa/team#41948)][]
-[Puppet CI milestone on Tor's GitLab][]
-[TPA-RFC-76: Puppet Merge Request workflow][]
-[2025Q1 Roadmap review][]
[TPA-RFC-73: Tails infra merge roadmap - Puppet Server]:https://gitlab.torproject.org/tpo/tpa/team/-/wikis/policy/tpa-rfc-73-tails-infra-merge-roadmap#puppet-server
[Merge Tails' and Tor's Puppet servers (tpo/tpa/team#41948)]:https://gitlab.torproject.org/tpo/tpa/team/-/issues/41948
[Puppet CI milestone on Tor's GitLab]:https://gitlab.torproject.org/groups/tpo/tpa/-/milestones/8#tab-issues
