Stop timers before expiration

If we don't stop them explicitly, the timers will not get garbage collected
until they timeout:
https://medium.com/@oboturov/golang-time-after-is-not-garbage-collected-4cbc94740082

Related to #40039
4 jobs for issue/40039 in 8 minutes and 1 second (queued for 3 seconds)
latest