Design experiment for evaluating HTTPS by default

As part of the Collaborative ResistancE to Web Surveillance (CREWS)'s project with UCL we are going to evaluate how effective HTTPS Everywhere EASE mode is. One element of this evaluation is a usability experiment to test EASE-mode and variations on a sample of users, and assess how well it performs. This ticket tracks work on designing this experiment.

One of the most important results from the experiment will be whether the design leads users to act appropriately in response to the notification that an upgrade to HTTPS failed. However, what is the right action that a user should take? There are still HTTP-only websites and so there are legitimate reasons for a user to proceed past the warning, but how should a user make this decision?

Some initial thoughts on the security of HTTP-downgrades are

• If the user has visited this website before with no warning (i.e. it previously accepted HTTPS) then a downgrade to HTTP is very suspicious
• If the website is from a large organisation they are likely to have resources to set up HTTPS therefore an HTTP downgrade is more suspicious than if it was a small organisation

Other questions could include:

• How distracting is the EASE address bar (which shows the extension URL) or is this not relevant because a production version won't do this?
• Should users act differently seeing this error message in Tor Browser vs. their usual browser vs. accessing an onion service?

What other evaluation criteria should there be for EASE mode?