Skip to content

GitLab

GitLab is used only for code review, issue tracking and project management. Canonical locations for source code are still https://gitweb.torproject.org/ https://git.torproject.org/ and git-rw.torproject.org.

Open
Created by anarcat@anarcat😸Owner0 of 10 tasks completed0/10 tasks

each node should have only one class in Hiera, which should be a role

We have some parts of our infrastructure that assume each node has only a single role applied to it. Prometheus, in particular, assigns a class label to each node which is in turn used by the main Grafana dashboard to filter the main view per role (more or less). This leads to confusing entries like this:

snap-20200706T113140

Now there are two ways of fixing this issue:

  1. make sure we include only one class from Hiera, and that that class should be a role
  2. create a new "role" field in Hiera that gets included as a class and used for classification in Hiera, then add an auxiliary "class" field that is used in the current way.

I favor the former, but I suspect we'll need the latter. For example, the roles::fde class seems to be included on and off on various nodes, and it will not be possible to include it across the board until either all servers are converted to FDE (which is unlikely: Ganeti nodes are already backed by their parent's FDE) or the class be made smarter about how it works. Either way, this will probably not be fixed across the board - proliant is another such example of classes being used to apply hardware-specific workarounds.

In either case, Hiera already needs a cleanup. Here's a few ideas:

  • alberti: roles::static_source should be included in roles::ldapdb
  • archive-01.torproject.org: roles::archive_www_host should include roles::static_mirror and roles::fde
  • eugeni: make a roles::eugeni that includes profile::postfix::eugeni and profile::lists
  • mandos: include profile::fde (not roles::fde?) in profile::mandos (or make a role::mandos?)
  • polyanthum: merge profile::bridges into roles::bridges?
  • cleanup collector roles
  • cleanup static mirror roles
  • cleanup onionoo roles
  • figure out the roles::fde question
  • nevii: fold roles::letsencrypt into roles::dns_primary?

Here are some details of the cleanup steps...

The collector stuff needs cleaning up:

hiera/nodes/colchicifolium.torproject.org.yaml:  - roles::weblog_sink
hiera/nodes/colchicifolium.torproject.org.yaml:  - roles::collector_backend
hiera/nodes/colchicifolium.torproject.org.yaml:  - roles::collector
hiera/nodes/corsicum.torproject.org.yaml:  - roles::collector_backend
hiera/nodes/corsicum.torproject.org.yaml:  - roles::collector

The static mirror stuff also needs cleanup:

hiera/nodes/hetzner-hel1-03.torproject.org.yaml:  - roles::static_mirror_web
hiera/nodes/hetzner-hel1-03.torproject.org.yaml:  - roles::static_mirror_onion
hiera/nodes/listera.torproject.org.yaml:  - roles::static_mirror_web
hiera/nodes/listera.torproject.org.yaml:  - roles::static_mirror_onion
hiera/nodes/listera.torproject.org.yaml:  - profile::media
hiera/nodes/listera.torproject.org.yaml:  - proliant
hiera/nodes/meronense.torproject.org.yaml:  - profile::metrics
hiera/nodes/meronense.torproject.org.yaml:  - roles::weblog_source
hiera/nodes/staticiforme.torproject.org.yaml:  - roles::rbm_master
hiera/nodes/staticiforme.torproject.org.yaml:  - roles::static_source
hiera/nodes/staticiforme.torproject.org.yaml:  - roles::onion_master
hiera/nodes/staticiforme.torproject.org.yaml:  - roles::www_master
hiera/nodes/static-master-fsn.torproject.org.yaml:  - roles::static_master
hiera/nodes/web-cymru-01.torproject.org.yaml:  - roles::static_mirror_web
hiera/nodes/web-fsn-01.torproject.org.yaml:  - roles::static_mirror_web
hiera/nodes/web-fsn-02.torproject.org.yaml:  - roles::static_mirror_web

Same with onionoo:

hiera/nodes/onionbalance-01.torproject.org.yaml:  - onion::balance
hiera/nodes/onionoo-backend-01.torproject.org.yaml:  - roles::onionoo_backend
hiera/nodes/onionoo-backend-02.torproject.org.yaml:  - roles::onionoo_backend
hiera/nodes/onionoo-frontend-01.torproject.org.yaml:  - roles::onionoo_frontend
hiera/nodes/onionoo-frontend-01.torproject.org.yaml:  - roles::onionoo_hidden
hiera/nodes/onionoo-frontend-02.torproject.org.yaml:  - roles::onionoo_frontend
hiera/nodes/onionoo-frontend-02.torproject.org.yaml:  - roles::onionoo_hidden
hiera/nodes/oo-hetzner-03.torproject.org.yaml:  - roles::onionoo_backend_old

Those are problematic and might be solved only with solution (2):

hiera/nodes/cache01.torproject.org.yaml:  - roles::cache
hiera/nodes/cache-02.torproject.org.yaml:  - roles::cache
hiera/nodes/cache-02.torproject.org.yaml:  - roles::fde
hiera/nodes/hetzner-hel1-02.torproject.org.yaml:  - bind::secondary
hiera/nodes/hetzner-hel1-02.torproject.org.yaml:  - roles::fde
hiera/nodes/hetzner-nbg1-02.torproject.org.yaml:  - roles::monitoring::external
hiera/nodes/hetzner-nbg1-02.torproject.org.yaml:  - roles::fde
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information