With every overlapped send or receive operation, it is possible that the data buffer submitted will be locked. When memory is locked, it cannot be paged out of physical memory. The operating system imposes a limit on the amount of memory that can be locked. When this limit is reached, the overlapped operations will fail with the WSAENOBUFS error.
}}}
But Tor doesn't use overlapped IO: it uses the select() system call to
learn when sockets are available for reading or writing, and then uses
non-blocking writes and reads to send and receive data.
So our theory is that when we send() and recv(), Windows copies
the contents of the buffer into a kernel buffer. If we send or recv
too much at once, Windows runs out of kernel buffer space.
Our current plan is that we need to abandon select() on Windows in favor
of overlapped IO. This involves three steps. Step one is to
add overlapped IO support to libevent. (Libevent already has a notion of
a buffer api, so we could extend that.) Step two is to change the way
Tor calls OpenSSL, so it operates on local buffers rather than interacting
with the network itself (presumably using recv and send). The third step
is to change Tor's networking loop to use libevent's buffer API rather than
the socket API. If you'd like to help with any of these steps, let us know!
Another guess is that the loop around select() is buggy in the
Windows libevent implementation. Tor is the only high-performance user of
libevent on Windows as far as we know, so this is quite possible. Check
You can try increasing the priority of Tor, Privoxy, and Vidalia in Taskmanager by hitting CTRL-ALT-DEL, going to the processes tab, and right clicking on each process and changing the priority to "Above normal". You can use Prio to make this automatic every time you start Tor.
You can also screw with the registry:
The following registry entries have been shown to mitigate the buffer
issues to varying degrees of success. As always, if you do not understand
the Windows Registry, and Reg``Edit, do not attempt these modifications.
Your mileage may vary.
At least one user has reported success by following the instructions
from [[http://web.ircsystems.net/codemastr/bufspace.html]]:
{{{
To do this go to Start, Run and type regedit. In the left pane navigate to
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters once
there, you must create the entry TcpNumConnections. To do this, right click in
the right pane and choose new from the menu and select DWORD Value. Give it the
name TcpNumConnections. Then right click it and select modify and enter a value
of 800. Then restart your computer.
}}}
There are a few TCP related registry entries that potentially manipulate
the internal buffer size available for data to be passed through the
Setting this to "1" is another option as it doesn't remove 12-bytes from every header for timestamp placement.
However, Tor seems to have lots of odd packet problems on an exit server (as shown by ethereal, lots of re-transmits,
lost ACKs, etc), and the "3" solution seemed to quiet these things down. (Only packet headers were captured during the tests, not actual data.)
HKLM\SYSTEM\Current``Control``Set\Services\Tcpip\Parameters\Sack``Opts="1" is another helpful setting.
An experimental feature recently added to Tor that constrains the send and receive socket buffer sizes may also reduce or alleviate this problem. If your Tor version supports it, try the following option in your configuration:
{{{
ConstrainedSockets 1
}}}
= Some more data points =
It appears that a system with 384MB of ram or greater, a fresh install of Win XP Home, fully patched
via Windows Update, and solely running a Tor exit server does not experience
these problems. This is true for both 0.1.0.16-stable and 0.1.1.12-alpha
versions of Tor. The configuration of tor is a simple exit server with no bandwidth limits, burst restrictions, nor hibernation.
We continue to debug this issue. Recent tests show that total available ram at boot time correlates with the creation of the [WSAENOBUFS] error. The amount of memory available to the system was configured via the C:\boot.ini option of ''/MAXMEM=###''. The results are as follows:
*At ''/MAXMEM=128'', simply starting up the tor server was enough to create a [WSAENOBUFS] error.
*At ''/MAXMEM=256'', the tor server did create a [WSAENOBUFS] error. Time varied from 2-5 hours.
*At ''/MAXMEM=384'', the tor server did not create a [WSAENOBUFS] error after 6 hours.
*At ''/MAXMEM=512'', the tor server did not create a [WSAENOBUFS] error after 6 hours. Further investigation is needed at this memory level.
*At ''/MAXMEM=1024'', the tor server did not create a [WSAENOBUFS] error after 48 hours.
We've learned that Windows does allocate large chunks of memory per socket on connect. See this graphic of [[http://img.photobucket.com/albums/v16/yoitsmeremember/charts/macroshaft.png|Non-Paged Pool Behavior]] in Win XP. It appears we are consuming against a hard limit, unable to be configured through registry settings. [[http://msdn.microsoft.com|MSDN]] articles refer to a hardcoded algorithm in non-server editions of Windows that determine non-paged pool size at boot. At this time, the memory factor, along with heavy network usage, appear to be the causes of the [WSAENOBUFS] error.
= Alternative solutions =
Virtualization doesn't help solve the underlying problem, but perhaps helps build the installed base. For lateral thinkers, VMWare Player (available at no cost) can be used by Windows users to run Tor on Linux. In particular the Browser Appliance [[http://www.vmware.com/vmtn/appliances/|available here]] might be a good starting point for a web client. There are many other [[http://www.vmware.com/vmtn/|VMWare Appliances]] which may also be easily modified to use Tor.
The [[http://janusvm.com/|JanusVM]] appliance provides a transparent proxy using Tor on Linux inside VMWare. This could also be used for acting as a server with the usual configuration applied to the following files at console:
*Configure the JanusVM to use a static IP address instead of DHCP using the menu.
*Edit the /etc/tor/torrc file with desired Tor Server settings.
*Modify firewall rules in /etc/init.d/janus.routing-with-tor to accept incoming requests on the ports required.
*Reboot the virtual machine.
Please consider the [[../OperationalSecurity| Operational Security]] requirements of running a Tor server before deploying on a VM just as you would for any other type of host.
