diff --git a/howto/backup.md b/howto/backup.md index 1d7db177c7219fa872ca90676d340c51a33ac84c..7a25b2eadd3ed4a9ce6c5651fe0686876be4603f 100644 --- a/howto/backup.md +++ b/howto/backup.md @@ -791,6 +791,44 @@ resolved the problem and the warning went away. It is assumed the problem will not return on the next job run. See [issue 40110](https://gitlab.torproject.org/tpo/tpa/team/-/issues/40110) for one example of this problem. +### Bacula GDB traceback / Connection refused / Cannot assign requested address: Retrying + +If you get an email from the directory stating that it can't connect +to the file server on a machine: + + 09-Mar 04:45 bacula-director-01.torproject.org-dir JobId 154835: Fatal error: bsockcore.c:209 Unable to connect to Client: scw-arm-par-01.torproject.org-fd on scw-arm-par-01.torproject.org:9102. ERR=Connection refused + +You can even receive an error like this: + +> root@forrestii.torproject.org (1 mins. ago) (rapports root tor) +> Subject: Bacula GDB traceback of bacula-fd on forrestii +> To: root@forrestii.torproject.org +> Date: Thu, 26 Mar 2020 00:31:44 +0000 +> +> /usr/sbin/btraceback: 60: /usr/sbin/btraceback: gdb: not found + +In any case, go on the affected server (in the first case, +`scw-arm-par-01.torproject.org`) and look at the `bacula-fd.service`: + + service bacula-fd status + +If you see an error like: + + Warning: Cannot bind port 9102: ERR=Cannot assign requested address: Retrying ... + +It's Bacula that's being a bit silly and failing to bind on the +external interface. It might be an incorrect `/etc/hosts`. This +particularly happens "in the cloud", where IP addresses are in the +RFC1918 space and change unpredictably. + +In the above case, it was simply a matter of adding the IPv4 and IPv6 +addresses to `/etc/hosts`, and restarting bacula-fd: + + vi /etc/hosts + service bacula-fd restart + +The GDB errors were documented in [issue 33732](https://gitlab.torproject.org/tpo/tpa/team/-/issues/33732). + ## Disaster recovery <a name="Restoring-the-directory-server"></a>