some websites block requests by HTTP User-Agent
Some websites will use the HTTP User-Agent field to determine whether the browser is allowed to visit. Apparently, this is done in the name of "security," with the assumption that "insecure" browsers should not be allowed to visit the site. (Probably, we should not assume that this has anything to do with security per se; perhaps it is really about correctness.)
The approach is neither necessary nor sufficient to achieve the objectives of the site operators. It is unnecessary because web standards define how browsers ought to behave, and any correctness should be determined by adherence to the standards, not by whether the name of the browser in question happens to be on some list. It is insufficient because circumventing the filter is trivial and can be done simply by changing the HTTP User-Agent, which users of Tor Browser can edit by editing general.useragent.override
on the about:config
page.
The default User-Agent that ships with Tor Browser appears to be:
Mozilla/5.0 (Windows NT 6.1; rv:52.0) Gecko/20100101 Firefox/52.0
This seems to work well if we want to appear to be using Firefox. However, sometimes Firefox is not on the approved list for websites such as those described above. (At least one website approves Safari and Chrome while rejecting IE and Firefox.)
Browser-Info provides a list of popular HTTP User-Agents, and choosing from this list we can configure Tor Browser to appear to be Safari by changing general.useragent.override
to:
Mozilla/5.0 (Windows NT 6.0) AppleWebKit/535.1 (KHTML, like Gecko) Chrome/13.0.782.112 Safari/535.1
Web users who do not value privacy may indeed have the option, inconvenient as it may be, to switch to a browser that satisfies the requirements of the site. Tor users do not have such an option, because there is only one Tor Browser (it happens to be based on Firefox).
We need to make it easier for everyday Tor users to circumvent filtering of this variety. Some possible suggestions:
- Maintain a list of popular User-Agents and provide an option in the drop-down onion menu on Tor Browser to choose which one to be for this site.
- Establish a Wiki page that allows users to report websites that block specific browsers by User-Agent, along with examples of User-Agent strings, if any, that work.
- Where appropriate, liaise with the websites in question, particularly if they are popular ones, to make sure that Tor Browser is on the list of suitable browsers.