Skip to content

GitLab

  • Projects
  • Groups
  • Snippets
  • Help
    • Loading...
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
Trac
Trac
  • Project overview
    • Project overview
    • Details
    • Activity
  • Issues 246
    • Issues 246
    • List
    • Boards
    • Labels
    • Service Desk
    • Milestones
  • Operations
    • Operations
    • Metrics
    • Incidents
  • Analytics
    • Analytics
    • Value Stream
  • Wiki
    • Wiki
  • Members
    • Members
  • Collapse sidebar
  • Activity
  • Create a new issue
  • Issue Boards

GitLab is used only for code review, issue tracking and project management. Canonical locations for source code are still https://gitweb.torproject.org/ https://git.torproject.org/ and git-rw.torproject.org.

  • Legacy
  • TracTrac
  • Issues
  • #11648

Closed (moved)
Open
Opened Apr 29, 2014 by Karsten Loesing@karsten

Problem parsing .z-compressed descriptors fetched via DirPort

I'm having trouble parsing compressed descriptors fetched via tor's DirPort. I don't know enough about .z compression to track down the problem though. Anyway, here's what I found:

Download consensus and all server descriptors from turtles:

curl http://76.73.17.194:9030/tor/status-vote/current/consensus.z > turtles-consensus.z
curl http://76.73.17.194:9030/tor/server/all.z > turtles-server-all.z

Attempt to parse compressed consensus using Python's zlib:

Python 2.7.5 (default, Aug 25 2013, 00:04:04) 
[GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.0.68)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import zlib
>>> print len(zlib.decompress(open('turtles-consensus.z', 'rb').read()))
1085611
>>> print len(zlib.decompress(open('turtles-server-all.z', 'rb').read()))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
zlib.error: Error -5 while decompressing data: incomplete or truncated stream
>>> print len(zlib.decompressobj().decompress(open('turtles-server-all.z', 'rb').read()))
8932838

Decompressing the consensus works fine using zlib.decompress(), but decompressing the server descriptors does not. The only workaround I found was to explicitly create a decompressor using zlib.decompressobj(). AIUI, the difference between the two approaches is that the latter can handle partial content (cf. http://stackoverflow.com/questions/20620374/how-to-inflate-a-partial-zlib-file/20625078#20625078).

Does that mean tor sends partial content?

Cc'ing atagar and wfn, because we discussed this problem a year ago: https://lists.torproject.org/pipermail/tor-dev/2013-May/004924.html

To upload designs, you'll need to enable LFS and have admin enable hashed storage. More information
Assignee
Assign to
Tor: 0.2.5.x-final
Milestone
Tor: 0.2.5.x-final
Assign milestone
Time tracking
None
Due date
None
Reference: legacy/trac#11648