... | @@ -749,6 +749,42 @@ happened between 6:26 and 13:57 on that day. |
... | @@ -749,6 +749,42 @@ happened between 6:26 and 13:57 on that day. |
|
be performed. See [Running a full
|
|
be performed. See [Running a full
|
|
backup](#running-a-full-backup) above.
|
|
backup](#running-a-full-backup) above.
|
|
|
|
|
|
|
|
Here is an example of the second case above, and recovery:
|
|
|
|
|
|
|
|
root@bungei:~# cd /srv/backups/pg/meronense
|
|
|
|
root@bungei:/srv/backups/pg/meronense# /usr/lib/nagios/plugins/dsa-check-backuppg -e -v
|
|
|
|
[meronense, main] WAL-MISSING-AFTER: meronense/main.WAL.00000001000005A900000073
|
|
|
|
[meronense, main] NO-BASE: no base backup found?
|
|
|
|
[meronense, main] NO-BACKUP: no backups! (no .backup files found)
|
|
|
|
NOT-CONFIGURED: meronense-11
|
|
|
|
IGNORED: .nobackup
|
|
|
|
IGNORED: lost+found
|
|
|
|
IGNORED: BASE-meronense.torproject.org:5432-G0uKzfYhtH.tar.gz
|
|
|
|
root@bungei:/srv/backups/pg/meronense# ls -altr
|
|
|
|
total 98312
|
|
|
|
-rw------- 1 torbackup torbackup 16777216 Jun 23 13:49 main.WAL.00000001000005A90000006F
|
|
|
|
-rw------- 1 torbackup torbackup 16777216 Jun 23 13:49 main.WAL.00000001000005A900000070
|
|
|
|
-rw------- 1 torbackup torbackup 16777216 Jun 23 13:58 main.WAL.00000001000005A900000071
|
|
|
|
-rw------- 1 torbackup torbackup 16777216 Jun 23 13:59 main.WAL.00000001000005A900000072
|
|
|
|
-rw------- 1 torbackup torbackup 16777216 Jun 23 13:59 main.WAL.00000001000005A900000073
|
|
|
|
-rw------- 1 torbackup torbackup 16777216 Jun 23 14:14 main.WAL.00000001000005A90000007F
|
|
|
|
drwxr-xr-x 10 torbackup torbackup 4096 Jun 23 14:14 ..
|
|
|
|
drwxr-xr-x 2 torbackup torbackup 4096 Jun 23 14:14 .
|
|
|
|
root@bungei:/srv/backups/pg/meronense# rm main.WAL.00000001000005A90000006F main.WAL.00000001000005A900000070 main.WAL.00000001000005A900000071 main.WAL.00000001000005A900000072 main.WAL.00000001000005A900000073
|
|
|
|
root@bungei:/srv/backups/pg/meronense# /usr/lib/nagios/plugins/dsa-check-backuppg -e -v
|
|
|
|
[meronense, main] NO-BASE: no base backup found?
|
|
|
|
[meronense, main] NO-BACKUP: no backups! (no .backup files found)
|
|
|
|
NOT-CONFIGURED: meronense-11
|
|
|
|
IGNORED: .nobackup
|
|
|
|
IGNORED: lost+found
|
|
|
|
IGNORED: BASE-meronense.torproject.org:5432-G0uKzfYhtH.tar.gz
|
|
|
|
|
|
|
|
There was a gap between `main.WAL.00000001000005A900000073` and
|
|
|
|
`main.WAL.00000001000005A90000007F` so everything up to and
|
|
|
|
including the former were removed by hand. Then a full backup was
|
|
|
|
performed. The reason why the BASE backup was missing is this was
|
|
|
|
following a failed upgrade (see [tpo/tpa/team#40809](https://gitlab.torproject.org/tpo/tpa/team/-/issues/40809)).
|
|
|
|
|
|
## Disaster recovery
|
|
## Disaster recovery
|
|
|
|
|
|
If all fails, we need to restore PostgreSQL from backups. We have two
|
|
If all fails, we need to restore PostgreSQL from backups. We have two
|
... | | ... | |