add new (OSUOSL) runners to remove load on ci-runner-01
Over the last week, the storage on ci-runner-01
filled up to 100% causing lots of CI job failures. Yesterday this happened and the runner was paused, but multiple shadow sims had been started just earlier in the day, so those runners weren't available to pick up jobs. When I looked, I saw two dozen jobs pending (waiting to be picked up by a runner) so I logged on to ci-runner-01
and ran the usual docker prune
commands which freed up space on /srv
and unpaused the runner.
I think deploying a new runner on the gnt-chi cluster would be a good option as we could easily dedicate 1.8TB of disk space to it.
Edited by anarcat