Skip to content

GitLab

  • Menu
Projects Groups Snippets
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
  • S Snowflake
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 70
    • Issues 70
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 6
    • Merge requests 6
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • The Tor Project
  • Anti-censorship
  • Pluggable Transports
  • Snowflake
  • Issues
  • #33367

Closed
Open
Created Feb 18, 2020 by David Fifield@dcfOwner

Goroutine leak in websocketconn

Thinking about legacy/trac#33364 (moved), I found that snowflake-server is chewing a lot of memory. It may be some memory leak or something.

$ top -o%MEM
  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
26910 debian-+  20   0 1916628 1.522g      0 S   0.0 77.8  58:51.37 snowflake-serve

The memory use seems to be inhibiting other processes. runsvdir puts status messages in its own argv so you can inspect them with ps. Currently it's reflecting xz not being able to allocate memory to compress logs:

$ ps ax | grep runsvdir
 1358 ?        Ss    94:01 runsvdir -P /etc/service log: locate memory \
svlogd: warning: processor failed, restart: /home/snowflake-proxy/snowflake-proxy-standalone-17h.log.d xz: (stdin): Cannot allocate memory \
svlogd: warning: processor failed, restart: /home/snowflake-proxy/snowflake-proxy-standalone-17h.log.d xz: (stdin): Cannot allocate memory \
svlogd: warning: processor failed, restart: /home/snowflake-proxy/snowflake-proxy-standalone-17h.log.d 

I even got it just now trying to run a diagnostic command (it doesn't always happen):

$ ps ax | grep standal
-bash: fork: Cannot allocate memory

In the short term, looks like we need to restart the server. Then we need to figure out what's causing it to use so much memory.

The server was last restarted 2020-02-10 18:57 (one week ago) at ca9ae12c383405bc9a755e1bc902e9755495c1f1 for legacy/trac#32964 (moved).

Assignee
Assign to
Time tracking