Skip to content

GitLab

  • Projects
  • Groups
  • Snippets
  • Help
    • Loading...
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
Trac
Trac
  • Project overview
    • Project overview
    • Details
    • Activity
  • Issues 246
    • Issues 246
    • List
    • Boards
    • Labels
    • Service Desk
    • Milestones
  • Operations
    • Operations
    • Metrics
    • Incidents
  • Analytics
    • Analytics
    • Value Stream
  • Wiki
    • Wiki
  • Members
    • Members
  • Collapse sidebar
  • Activity
  • Create a new issue
  • Issue Boards

GitLab is used only for code review, issue tracking and project management. Canonical locations for source code are still https://gitweb.torproject.org/ https://git.torproject.org/ and git-rw.torproject.org.

  • Legacy
  • TracTrac
  • Issues
  • #22141

Closed (moved)
Open
Opened May 03, 2017 by Karsten Loesing@karsten

Deprecate `DescriptorFile` and add relevant information to `Descriptor`

I'd want us to implement #20395 (moved) where we'd be able to handle much larger descriptor files without copying all contents to memory before even looking at them. But I realized that DescriptorFile#getDescriptors() makes this rather pointless. If we need to keep a list with all parsed descriptors in memory, each containing raw contents of some sort (see #22140 (moved)), then what's the point of avoiding to read complete files to memory?

One way to fix this is to deprecate DescriptorFile and add all relevant information to Descriptor. And then DescriptorReader would return an Iterator<Descriptor>, internally backed by BlockingIteratorImpl<Descriptor>, rather than an Iterator<DescriptorFile>. Sounds easy!

Here's a catch though: DescriptorFile#getException() returns "the first exception that was thrown when reading this file or parsing its content". If we'd move this method to Descriptor, would we set an I/O exception to the first descriptor in that file, to all of them, or maybe to none of them? Turns out we don't have to worry too much about this. The only code that actually uses DescriptorFile#getException() is Onionoo's DescriptorQueue, and all it does is log the exception. We could just do the same.

What else could go wrong?

To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information
Assignee
Assign to
metrics-lib 1.9.0
Milestone
metrics-lib 1.9.0
Assign milestone
Time tracking
None
Due date
None
Reference: legacy/trac#22141