add new (OSUOSL) runners to remove load on ci-runner-01
Over the last week, the storage on
ci-runner-01 filled up to 100% causing lots of CI job failures. Yesterday this happened and the runner was paused, but multiple shadow sims had been started just earlier in the day, so those runners weren't available to pick up jobs. When I looked, I saw two dozen jobs pending (waiting to be picked up by a runner) so I logged on to
ci-runner-01 and ran the usual
docker prune commands which freed up space on
/srv and unpaused the runner.
I think deploying a new runner on the gnt-chi cluster would be a good option as we could easily dedicate 1.8TB of disk space to it.