Updated robots.txt file to disallow crawling as in ticket #29565.

The server log scrubbing code from ticket #21304 is now refactored into
a safelog package, along with the appropriate tests

This is related to the proxy-go deadlock bug #25688. If a client doesn't
do anything with the SDP answer, a token will get lost. Added a timeout
after a minute that checks the PeerConnection state and destroys the
peer connection and returns a token if did not yet succeed

This is a fix for the proxy-go deadlock bug (ticket #25688). The
assumption that OnIceComplete is always followed by a successful
connection where OnDataChannel has been called turns out not to occur in
practice. OnICEComplete looks like it is being deprecated in other
libraries anyway, so it's safer to just remove it.

Removed raw bytes of ICE answers from broker logs and the SDP stanzas
from the client logs

Modified regular expressions to not scrub fingerprints, but catch all
instances of IPv4 and IPv6 addresses. Expanded test cases with those
suggested by dcf.

Log scrubber now scrubs addresses even in the case where they are split
across calls to the scrubber's output io.Writer.

Added test cases to test that the writer behaves correctly across split
lines.

It is now more readable, and the regexp's are only compiled once

IPv6 regexes didn't need to be that precise, added more tests for
edge-cases.

Added a scrubber that takes all logging output to the standard logger
and passes through a series of regular expressions to replace IP
addresses with safe strings (e.g., X.X.X.X:443).

Ensure server logs to stdout are also scrubbed

To facilitate local testing.

Without this, running with non-dummy rate limiter (e.g. ?ratelimit=1000)
would try to add undefined to a number resulting in NaN.

Same as the other direction.

It doesn't really help to have a bunch TLS application records
interpreted as a string written to the console.

The binaryType can be "arraybuffer" or "blob", and "blob" is the
default. The code is only aware of "arraybuffer": I discovered a problem
while running snowflake.html in debug mode; this code fails:
    if DEBUG
      # Go sends only raw bytes...
      if '[object ArrayBuffer]' == recv.toString()
        bytes = new Uint8Array recv
        line = String.fromCharCode.apply(null, bytes)
      line = line.trim()
      log 'WebRTC --> websocket data: ' + line
with the error:
	TypeError: line.trim is not a function[Learn More] snowflake.js:497:16
because recv is of type Blob, not ArrayBuffer.

Despite the unexpected type, the code seemed to work as expected when
not in debug mode. Though the two types provide different interfaces,
they are both valid to pass on to WebSocket.send. The only other thing
we did with it was try to read the .length member for rate-limiting
purposes:
        @rateLimit.update chunk.length
but .length is incorrect for either type: Blob uses .size and
ArrayBuffer uses .byteLength. It worked anyway, because
DummyRateLimit.update doesn't actually look at its argument.

We were already setting binaryType="arraybuffer" for WebSocket
connections.

Maybe we'll able to remove this in the far future.

Addendum to #28727.

These could be misused to cause Snowflake proxies to attempt connect to
unsuspecting third parties.
https://bugs.torproject.org/28727

ReferenceError: age is not defined	snowflake.js:265:7
        BucketRateLimit.prototype.when	snowflake/proxy/build/snowflake.js:265:7
        ProxyPair.prototype.flush	snowflake/proxy/build/snowflake.js:558:63
        bind/<	snowflake/proxy/build/snowflake.js:10:56
        ProxyPair.prototype.onClientToRelayMessage	snowflake/proxy/build/snowflake.js:495:14
        bind/<	snowflake/proxy/build/snowflake.js:10:56

I got the warning:
	Expected declaration but found ‘/’.  Skipped to next declaration.

This was mainly a matter of more complete mocking.

You can nest a "describe" in a "describe":
  describe
    describe
      it

But you can't nest an "it" in an "it":
  describe
    it
      it

The nested "it"s were not getting run (or getting run, but their output
ignored, I'm not sure).

Before this change:
	41 specs, 0 failures
After:
	44 specs, 5 failures