archive trac.torproject.org to the internet archive
As discussed in #31690 (closed), we're going to redirect bugs.torproject.org (done) and trac.torproject.org (#34373 (closed)) to gitlab eventually. But in the meantime, we want to create a full archive of trac.torproject.org on the internet archive.
Activity
jobs were created in ArchiveBot (c4xu3x5idgipxj07hlpog418a and bpu6j3ucrv87g4aix1zdrhb6k) to archive (respectively) all the tickets individually and crawl the entire site.
This should ensure we have a good, up to date copy of trac whatever happens from here on. Last time, the latter crawl took almost a week.
added Doing label
mentioned in issue #31690 (closed)
-
16:18:00 <Major> anarcat: Your job for https://paste.anarc.at/publish/2020-06-17/trac.torproject.org-tickets-1-40000-final.txt has finished.this means all tickets are already individually archived on archive.org. the dump will show up here shortly:
https://archive.fart.website/archivebot/viewer/?q=trac.torproject.org
the other crawl will take a few more days.
-
crawl still going, we can see progress in this dashboard:
http://dashboard.at.ninjawedding.org/?showNicks=1
enter "trac" (or the job hash) in the search field to find the job.
right now it's at
20,111.1 MB in 71,992 resp. at 2.4/s, 316,267 in q. -
the archive is now complete. it will take a few days to propagate to the view and the wayback machine, but I am confident we have the integrality of Trac in the internet archive for the forseeable future.
the currently known archives are:
- june 2019 ticket crawl: 6h30, 29892 files, 1.9 GiB
- june 2020 ticket crawl: 4h30, 33582 files, 1.9GiB
- june 2019 and 2020 full crawls: 5 days, 7h30, 732488 files, 105.4 GiB; 5 days, 16h10, 837100 files, 137.6 GiB.
I documented this in the GitLab migration docs.
So this is done!
Edited by anarcat