Disk almost full on backup-storage-01

First diagnostic 

05:02 -ALERTOR3:#tor-alerts- DiskWillFillSoon [firing] Disk /srv/backups/bacula on backup-storage-01.torproject.org is filling up
Logs and dumps, click to expand biggest backup sets on the volume: 
--- /srv/backups/bacula ----------------------------------------------------------------------------------------------------------------------------------------------------------------------
   14.2 TiB [###########################] /archive-01.torproject.org
    6.4 TiB [############               ] /colchicifolium.torproject.org
    2.0 TiB [###                        ] /metrics-store-01.torproject.org
    1.4 TiB [##                         ] /lists-01.torproject.org                                                                                                                            
    1.2 TiB [##                         ] /tbb-nightlies-master.torproject.org
    0.9 TiB [#                          ] /perdulce.torproject.org
  475.3 GiB [                           ] /meronense.torproject.org
  299.9 GiB [                           ] /staticiforme.torproject.org
  153.2 GiB [                           ] /media-01.torproject.org
  131.0 GiB [                           ] /gayi.torproject.org
  130.2 GiB [                           ] /polyanthum.torproject.org
  105.1 GiB [                           ] /rude.torproject.org

Current status

Roles

  • Lead: unless otherwise noted, the issue assignee
  • Operations:
  • Communications:
  • Planning:

Next steps

  • TBD

Dashboards

Post-mortem

Detailed post-mortem to fill in later, click to expand
  • Affected users:
  • Duration:
  • Status page link:
  • Report Status: not started

Timeline

Root cause analysis

What went well?

What could have gone better?

Recommendations and related issues