Crawlers slowing down our services
Hey,
Lately crawling is slowing down some of our services. This has been affecting trac and gitweb particularly. Maybe we should try to find out if requests come from a single crawler and see if we can limit the number of requests per amount of time, or just slow it down.