configure metricsdb-02

Hello, I would need a lil tweaking of metricsdb-02.

Specifically:

• Can I be given sudo powers like in the other net-health machines?
• Can we remove all services from metricsdb-01? I do not need these to run.
• Can I have a role to connect to the postgresql db on metricsdb-01 from metricsdb-02? I want to be able to copy tables over

Thanks!

added For Network Health Team Next lifecycle labels

assigned to @lelutin

marked this issue as related to #41969 (closed)

added Doing label and removed Next label

mentioned in issue #41969 (closed)

@hiro Your sudo to root access is now in place for metricsdb-02. Puppet is also now managing postgresql.

for the cross-server access to postgres, metricsdb-01 already permits all of the internet to connect to postgresql's port so we just need a postgres role on metricsdb-01 that'll let us connect and read data. that role needs read-only access to all objects in the 'parser' database, is that correct?

for the second point in the description, about removing services, I'm wondering if you were meaning that we should remove services from metricsdb-02 instead since those have never been used yet. so far on metricsdb-02 I've only removed apache2 and the "parser" service in order to make systemd units not crash. Should I find all that puppet installed when I setup the host as a copy of metricsdb-01 and remove all of those from metricsdb-02, except for postgresql?

Thanks for the sudo.

I would need a role that can read and write on the postgres parser db from metricsdb-02 to metricsdb-01.

I thought metricsdb-02 inherited all the services currently running on metricsdb-01 and I meant that we do not need those at the moment.

I can create the service running and then I can send you a MR for puppet if that's ok.

I would need a role that can read and write on the postgres parser db from metricsdb-02 to metricsdb-01.

OK thanks for the details. I'll look at how I can create that role now.

I thought metricsdb-02 inherited all the services currently running on metricsdb-01 and I meant that we do not need those at the moment.

that was indeed what I tried at first, e.g. just reusing the role used on metricsdb-01 for metricsdb-02. I did that mostly because the name is the same and it's related enough in terms of usage.. but puppet failed to correctly apply things and caused some conflicts for certain exported resources, so I walked back just enough that puppet won't fail anymore and monitoring is somewhat happy about the machine's state.

last week I removed apache2 entirely and the systemd units for the parser and geoip services. today to completely make sure that services would not be running, I've also removed the systemd units for metrics-api (which wasn't running), tagtor (also not running).

I did not remove debian packages (apart from apache2 which needed to be purged for puppet to stop re-configuring it), users, sudo rules for them and most files installed by puppet.

Particularly, I didn't remove the git checkouts that puppet placed in /srv, but feel free to remove any directories in that location at your convenience if your need to free up some space.

I can create the service running and then I can send you a MR for puppet if that's ok.

That would be perfect! thanks

@anarcat since this host is meant to only contain copies of data from metricsdb-01, I wonder if we can complete disable backups for this host. Just with a quick search through the puppet codebase I didn't see a way to disable backups for a host. Do you know off the top of your head if we have something for this?

added Needs Information label and removed Doing label

@gk

added Next label and removed Needs Information label

@anarcat I had moved this issue to Needs Information since we're waiting for @hiro 's MR. did I not use the right label? ...or maybe I should just close and wait for the MR to come in?

yeah, sorry, i probably messed up. somehow i thought hiro had answered here...

added Needs Information label and removed Next label

removed Needs Information label

@lelutin We do not need backup. Also regarding the MR, I think there was a misunderstanding. I won't be sending this right now, I need to figure out all the data ingestion, copy over and archive process. I think it might take me a few weeks. What do you think if we close this for now?

@hiro wrt backups: ok perfect that was my understanding as well. I can't disable bacula in our puppet code, but I have already disabled postgresql backups for metricsdb-02 so even though bacula runs it won't be sending much.

wrt the merge request: ok great, thanks the feedback on this. I'll close this issue then.

cheers!

closed

err wait.. reopening. I forgot one detail: I need to setup the role in metricsdb-01 that lets metricsdb-02 access the data

just sent puppet-control!15 to create the role and the network access permission. I couldn't find out how to grant access to the database easily. the module has postgresql::server::database_grant but it feels weird to add this to the generic role code for metricsdb.

reopened

@hiro the new postgresql role is now in place on metricsdb-01 and I've tested that I was able to connect to the parser database with that role from metricsdb-02 and then I could read from and write to tables in there.

Puppet is not managing the role's password so you can set it on metricsdb-01 with this as root:

sudo -u postgres psql
[...]
postgres=# \password metricsrw

Then to connect from metricsdb-02 you have to specify the database name as well since it's the only one accessible, e.g.:

metricsdb-02$ psql -h metricsdb-01.torproject.org -U metricsrw parser

Setting the issue to Needs Review. Feel free to close it if things work, or just make a sign and I'll close it

added Needs Review label

Thanks! @lelutin

Everything works just fine.

closed