Skip to content

GitLab

  • Projects
  • Groups
  • Snippets
  • Help
    • Loading...
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
Trac
Trac
  • Project overview
    • Project overview
    • Details
    • Activity
  • Issues 246
    • Issues 246
    • List
    • Boards
    • Labels
    • Service Desk
    • Milestones
  • Operations
    • Operations
    • Metrics
    • Incidents
  • Analytics
    • Analytics
    • Value Stream
  • Wiki
    • Wiki
  • Members
    • Members
  • Collapse sidebar
  • Activity
  • Create a new issue
  • Issue Boards

GitLab is used only for code review, issue tracking and project management. Canonical locations for source code are still https://gitweb.torproject.org/ https://git.torproject.org/ and git-rw.torproject.org.

  • Legacy
  • TracTrac
  • Issues
  • #5806

Closed (moved)
Open
Opened May 08, 2012 by Karsten Loesing@karsten

Investigate why metrics-db sometimes takes longer than 5 minutes to run

metrics-db runs once every hour and shouldn't take longer than a few minutes. Ideally, we should run it twice per hour to catch consensuses published at :30 of an hour (see #5504 (moved)). That's only possible if it reliably finishes in under 30 minutes, ideally under 5 minutes.

My current idea why metrics-db takes so long is that it implements a quite crappy algorithm to provide the last 3 days of data via rsync: enumerate all files in the current rsync/ directory and store their names and last modified times in memory, iterate over our various output directories, copy files that are missing in rsync/, and delete the ones in rsync/ that are older than 3 days. The problem is the step where we iterate over output directories, because files are not deleted automatically after a given time. Any change needs to be implemented for relay descriptors, bridge descriptors, bridge pool assignments, exit lists, etc.

This task took 1.5 points so far and may take another 2.5 points for a real fix.

To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information
Assignee
Assign to
None
Milestone
None
Assign milestone
Time tracking
None
Due date
None
Reference: legacy/trac#5806