Is the tool open source?
No, but it's written in Java.
Is the data collected made public?
Not unless you count their map. Data from a user's session, both clientside and serverside transcripts, are made available for download after the tests are completed. Before running the tests, a user may view an example result, clientside transcript, and serverside transcript.
Is the data format that is used for publication easy to interact with?
Is is sorted by timestamp, similarly to dmesg output. One thing that could make it easier to read in this format would be a translation between the client and server timestamps.
Are the methodologies explained?
The types of tests conducted are listed, and substantial effort is put into adding optional "more info" buttons which give explanations balanced for both laypersons and the technically-inclined.
Is the tool to be used by the general public?
If so, is the user warned of possible risks that he may incur when running the tool?
No, however risks for most users are incredibly low.
Does the data collected by the tool include potentially sensitive information?
What kind of tests does the tool perform?
If the user's JVM security settings allow all tests to be run, the tests extensively test network settings, bandwidth, and configuration, as well as possible proxies and captive portals, MITMs for certain protocols, firewalls and gateway settings, existence of NAT and uPnP devices, kernel settings such as UDP buffer size and clock drift, and the configuration of local DNS resolvers.
How accurate are the tests?
There appears to be minimal false positives, and information that could be erroneous is explained in the results.
What claims does the tool make?
They claim to only collect the information which is displayed to the user in the results.
Are the claims satisfied?
How does the reporting system work?
Several HTTP POSTs throughout the test procedure of the information to netalyzr.icsi.berkeley.edu. Both the web applet and the commandline client report in this way.
Is confidentiality and integrity of data being reported maintained?
Not at all. Plaintext over the wire.
What are it's strengths?
It's quite thorough in its tests of TCP, UDP, and DNS quirks.
What are it's weaknesses?
Java. And the HTTP POSTs.
What tests it performes
Tries 0x20 hack on HTTP headers and checks raw response for identical capitalization
checks for reordered HTTP headers
checks for changed/added/unsolicited HTTP headers, checks headers lengths
checks raw HTTP content length
checks if google 302 redirects to a different google page
check for "insert w3 info and url here>" and "Google" in response page
If a potential proxy was discovered via ICSI-Client-Addr, attempts an HTTP connection to the proxy and requests http://www.google.com, if a response comes back, check again for the ICSI-Client-Addr header
Does an HTTP GET for fake files of known size with .exe, .torrent, and .mp3 extensions, check for a fetching failure, an I/O exception, file corruption, checks the headers on the fetch's response, checks the size of the returned file
Checks that 404s are properly received for bogus urls: ./notfound/mode-/.html
HTTP GET to for an image with known size and locally calculated MD5sum with the following extra headers set: If-Modified-Since: If-None-Match: Pragma: no-cache Cache-Control: no-cache/no-store
If a proxy was previously discovered, first queries the proxy directly, and checks the response headers, and also the "Last-Modified" and "ETag" response headers, then compares MD5sums.
Checks for strong and weak Etag validation. Etags can be used for tracking purposes.
- Checks DNS by requesting the inet adress of a domain (constructed from $nonce..:) by name on the locally configured DNS resolver, then requests IP of domain with domain's returned inet address. Domain's actual expected IP is hardcoded.
- Checks DNS resolution for bogus domain ('return-false.abcd'), expects an UnknownHostException
- Checks DNS MTU by checking to see if ednspadding-0.xdnsmtu.$nonce..: is reachable, then making 513B, 12801B, and 1473B requests, then trying the maximum 4000B, then slowly decreasing to get the actual max if 4000B doesn't work.
- Tries to resolves each of these domains (we're first on the list?) twice, then gives up: "www.torproject.org", "www.meebo.com", "www.paypal.com", "www.google.com", "mail.google.com", "www.google-analytics.com", "encrypted.google.com", "www.yahoo.com", "mail.yahoo.com", "search.yahoo.com", "www.bing.com", "www.microsoft.com", "windowsupdate.microsoft.com", "mail.live.com", "www.bankofamerica.com", "www.chase.com", "chaseonline.chase.com", "www.wamu.com", "www.wellsfargo.com", "online.wellsfargo.com", "www.f-secure.com", "www.trendmicro.com", "www.secureworks.com", "www.citibank.com", "online.citibank.com", "www.citi.com", "www.citimortgage.com", "www.smithbarney.com", "ad.doubleclick.net", "smartzone.comcast.net", "partner.googleadservices.com", "wireless.att.com", "www.ticketmaster.com", "www.facebook.com", "www.schwab.com", "www.etrade.com", "us.etrade.com", "www.tdameritrade.com", "www.ameritrade.com", "www.abbey.co.uk", "www.alliance-leicester.co.uk", "www.amazon.com", "www.bankofscotland.co.uk", "www.bankofthewest.com", "www.barclays.co.uk", "www.capitalone.com", "www.careerbuilder.com", "www.cnn.com", "www.desjardins.com", "www.deutsche-bank.de", "www.e-gold.com", "www.ebay.com", "www.fdic.gov", "www.friendfinder.com", "www.halifax.co.uk", "www.hsbc.co.uk", "www.irs.gov", "www.jpmorganchase.com", "www.lloydstsb.com", "www.mbna.com", "www.mbna.net", "messenger.yahoo.com", "www.nationwide.co.uk", "www.networksolutions.com", "www.newegg.com", "www.nordea.fi", "www.orange.fr", "www.postbank.de", "www.rbs.co.uk", "www.sears.com", "www.sparkasse.de", "www.sterlingsavingsbank.com", "www.usbank.com", "www.verisign.com", "www.visa.com", "www.wachovia.com", "westernunion.com", "tinyurl.com", "bit.ly", "www.youtube.com", "www.twitter.com", "www.thepiratebay.org", "thepiratebay.org", "www.twitpic.com", "hootsuite.com", "appspot.com", "www.blogimg.jp", "blogpark.jp", "6park.com", "www.6park.com"
- Does most of the above DNS checks over TCP
- Makes CHAOS queries to attempt to discover anycast DNS root servers, see this paper. The list of hardcoded DNS root server IPs is: 18.104.22.168 (Verisign) 22.214.171.124 (BRoot) 126.96.36.199 (Cogent Communications) 188.8.131.52 (U. of Maryland) 184.108.40.206 (NASA) 220.127.116.11 (ISC) 18.104.22.168 (DoD) 22.214.171.124 (US ISC Headquarters) 126.96.36.199 (Netnod, Sweden) 188.8.131.52 (Verisign) 184.108.40.206 (KRoot) 220.127.116.11 (ICANN) 18.104.22.168 (U. of Tokyo)
- Checks for NAT DNS resolvers, and for 2wire devices, if found, checks for MITMing of DNS queries
- Checks for DNS resolver port randomization, for more info see this commit in my dnstamper branch of ooni, specifically the DNSTamperResolver class.
- Does a bunch of the same checks for IPv6 DNS
- Check if local DNS resolver (lDNSr) will accept exact, internal, and exteral glue entries. Check if lDNSr will try to resolve a CNAME for an NS
- Check if lDNSr passes 0x20 queries correctly
- Check if lDNSr uses ANY for A records
- Check if lDNSr will resolve AAAA records for IPv6only hosts
- Check if lDNSr support DNSSEC and/or EDNS, if so, then also check EDNS MTU
- Checks if lDNSr will handle different sized TXT records
- Checks if lDNSr wildcards NXDOMAIN errors by seeing if "www.yahoo.cmo" falsely returns as "www.yahoo.com"
- Checks if lDNSr DNS wildcarding extends to "typos" at the sld level
- Checks that random hostnames properly resolve to NXDOMAIN
- Check if DNS works over TCP
- If lDNSr accepts glue records, test creation of glue records with TTL 0 and 1
- Try to query an IPv6only DNS server
- Checks if SOA records can be retrieved
- Make queries to lDNSr for random hostnames which are as yet uncached, calculate median uncached lookup time, then re-request all of them again and calculate median cached lookup time
- Checks if lDNSr supports a sufficient ammount of DNS transactionID entropy, which lDNSr should be doing to avoid the subdomain cache poisoning attack.
- Checks if an external DNS proxy can be used, and if so, does all the previous DNS tests for that proxy
- Checks if non-DNS UDP packets can be sent over port 53
Sends useragent, language, encoding, charset, agentID, nonce
Tries direct TCP/IPv4 and TCP/IPv6 connections on "$TCP_ECHO_PORT" (?), 21, 22, 25, 53, 80, 110, 135, 139, 143, 161, 443, 465, 465, 585, 587, 993, 995, 1194, 1723, 5060, 6681, 9001
Sends info on client network interfaces: addrs, hostnames, inetaddrs, and available bandwidth
Checks for ability to send raw UDP, also checks path MTU holesby sending and receiving large (1471B) UDP packets
Checks for ability to send and recieve fragmented UDP packets, and for max MTU
Checks which ports UDP is available on, including common ports: 53 (DNS), 123 (NTP), 137 (NetBIOS), 138 (NetBIOS DGM), 500 (IKE), 1194 (OpenVPN), 1434 (Slammer), 1701 (L2 tunnels), 4500 (IPSec NAT), 5004 (RTP), 5005 (RTCP), 5060 (SIP), 7078 (VoIP), 7082 (VoIP), 9899 (SCTP), 27005 & 27015 (Steam)
Checks if the device in behind NAT
Builds a multicast socket on local inet and tries sending a broadcast datagram host discovery query
Tries to discover local uPnP devices, their NAT addressses, and location URLs
HTTP POST to a server over port 80 of the agentID, all uPnP devices, and the uPnP device descriptions
gets previous POST results and uses them to try to measure one-way-delay, compensating for clock drift
calculates kernel UDP buffer size
Checks and sends (over http:80 again) WAN info: connection type, IP address, WANPPP connection info and IP (if available), external IP
Does UDP bursts and measures loss
server sends back an ICSI-Client-Header, checks for match against external IP, and != then that client is potential proxy
checks send/recv UDP MTU (again, I believe?), allowed fragmentation size, and does a binary search to attempt to find the bottleneck IP
Checks if traceroutes work, and tries to identify bad hops for IPv4 and IPv6
Tries to send ICMP packets above the max size (~1KB)