Skip to content

GitLab

  • Projects
  • Groups
  • Snippets
  • Help
    • Loading...
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
Trac
Trac
  • Project overview
    • Project overview
    • Details
    • Activity
  • Issues 246
    • Issues 246
    • List
    • Boards
    • Labels
    • Service Desk
    • Milestones
  • Operations
    • Operations
    • Metrics
    • Incidents
  • Analytics
    • Analytics
    • Value Stream
  • Wiki
    • Wiki
  • Members
    • Members
  • Activity
  • Create a new issue
  • Issue Boards
Collapse sidebar

GitLab is used only for code review, issue tracking and project management. Canonical locations for source code are still https://gitweb.torproject.org/ https://git.torproject.org/ and git-rw.torproject.org.

  • Legacy
  • TracTrac
  • Issues
  • #25227

Closed (moved)
Open
Created Feb 13, 2018 by teor@teor

Avoid storing all Tor nodes in RAM

<teor4>	ahf: I've been thinking about reducing Tor's RAM usage by sampling from each consensus, rather than keeping it all in RAM
<teor4>	there are ways to stream the consensus, pick a weighted sample, and just keep those nodes
<teor4>	https://en.wikipedia.org/wiki/Reservoir_sampling
<teor4>	it would really help the iOS VPN, and other embedded impls
<teor4>	The catch is that if you run out of sampled nodes, you need to re-parse the consensus
<+ahf>	teor4: i had been thinking a bit in the back of my head if we could refactor the code that is used for accessing the consensus to be a bit more pluggable to experiment with different ways of backing them
<+ahf>	teor4: on ios i think we can mmap() files without them adding to the memory limit of the application too
<teor4>	that would be helpful, because we could only keep the selected node queues in RAM
<teor4>	if we got the queue size right, we could regenerate them once per consensus
<+ahf>	do you want to create a ticket with this? maybe add sponsor8-can to the keywords

We could be smart about node selection, and just store some weights for each node. And then when we use too much RAM for nodes, we could evict older nodes.

To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information
Assignee
Assign to
Tor: unspecified
Milestone
Tor: unspecified
Assign milestone
Time tracking
None
Due date
None