proxy-go sometimes gets into a 100+% CPU state
proxy-go sometimes works itself into a state where it is still running and working, but using more than 100% CPU. I have had it happen locally a couple of times while testing turbotunnel stuff, and it's currently happening with proxy-go-restartless:
$ top
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
13844 snowfla+ 20 0 551292 320692 8844 R 161.1 15.6 129356:18 proxy-go
Or looking at single threads:
$ top -H
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
24628 snowfla+ 20 0 551292 320692 8844 R 39.7 15.6 15219:01 proxy-go
13844 snowfla+ 20 0 551292 320692 8844 R 35.4 15.6 15431:52 proxy-go
1637 snowfla+ 20 0 551292 320692 8844 R 34.8 15.6 16057:40 proxy-go
13848 snowfla+ 20 0 551292 320692 8844 S 27.5 15.6 13669:02 proxy-go
13846 snowfla+ 20 0 551292 320692 8844 S 22.5 15.6 17021:57 proxy-go
I caught it once and attached to the process with GDB, but didn't know what to make of it. thread apply all bt
seemed to show all the threads being somewhere in the Go runtime; the thread that wasn't was not one of the threads using a lot of CPU. (Matching up the PID
field from top -H
with the LWP
identifiers in gdb.)
I had the idea to make proxy-go emit profiling output, and then exmine the call chain that was resulting the in the most CPU using profiling tools. A patch to do that is proxy-go-profile.patch. But I haven't been able to reproduce the high CPU usage yet.