howto/gitlab.md

@@ -961,6 +961,34 @@ Delivered mail 641c797273ba1_86be948d03829@gitlab-02.mail (7.2ms)
 
 [issue 139]: https://gitlab.torproject.org/tpo/tpa/gitlab/-/issues/139
 
+### Sidekiq jobs stuck
+
+If merge requests don't display properly, email notifications don't go
+out, and, in general, GitLab is being weird, it could be Sidekiq
+having trouble. You are likely going to see a `SidekiqQueueSize` alert
+that looks like this:
+
+    Sidekiq queue default on gitlab-02.torproject.org is too large
+
+The solution to this is unclear. During one incident
+([tpo/tpa/team#42218](https://gitlab.torproject.org/tpo/tpa/team/-/issues/42218)), the server was running out of disk space
+(but did *not* actually run out completely, it still had about 1.5GB
+of disk available), so the disk was resized, GitLab was upgraded, and
+the server rebooted a couple of times. Then Sidekiq was able to go
+through its backlog in a couple of minutes and service was restored.
+
+Look for lack of disk space, and in all of GitLab's logs:
+
+    tail -f /var/log/gitlab/*.log
+
+Try to restart GitLab:
+
+    gitlab-ctl restart
+
+... or rebooting the server.
+
+Update this section with future incidents as you find them.
+
 ### Gitlab registry troubleshooting
 
 If something goes with the GitLab Registry feature, you should first