TorBrowser 5.0a3 acting very odd with e.g. Twitter due to IndexedDB

I'm experiencing numerous problems with Twitter's web interface with the latest alpha version of TorBrowser. I haven't seen this on other websites so I'll enumerate issues here:

• keyboard shortcuts (e.g., opening a tab with ctrl-t, saving a bookmark with ctrl-d) don't work while in the twitter tab but work elsewhere
• images aren't displayed
• trending topics & follow suggestions aren't displayed
• twitter doesn't automatically load/queue new tweets

All of these sound like JS issues, but even whitelisting twitter.com and twimg.com in NoScript didn't seem to make a difference.

added GeorgKoppen201507R MikePerry201507 TorBrowserTeam201507R component::applications/tor browser ff38-esr owner::mikeperry priority::high resolution::fixed severity::normal status::closed tbb-5.0a4 tbb-usability-website type::defect labels

I can confirm this is a problem for me, too.

• pictures in Tweets are not shown
• I'm unable to show the context menu via right clicking in the Twitter site
• old Tweets are not loading when scrolling to the end of the shown tweets

I'm on Windows using TB 5.0a3 (based on Mozilla Firefox 38.1.0).

As an addition, I'm not using NoScript and disabling all addons doesn't solve these problems!

Trac:
Username: Diapolo

I too can confirm trouble with Twitter, on a 32bit linux system.

In addition, another JS-heavy site that I'm using for banking doesn't work any more. Specifically, "drop down menus" don't drop down any more on mouse over. (Sorry for inprecise language.)

Trac:
Keywords: N/A deleted, ff38-esr, TorBrowserTeam201507, tbb-usability-website, tbb-5.0a added
Priority: normal to major

This appears to due to an error with how a script that is crucial to twitter's page handling is checking for/attempting to use IndexedDB (which we disable).

If I set the about:config pref 'dom.indexedDB.enabled' to true, twitter appears to work much better for me. Can anyone confirm that this fixes all issues for them, or if any still remain?

Replying to mikeperry:

If I set the about:config pref 'dom.indexedDB.enabled' to true, twitter appears to work much better for me. Can anyone confirm that this fixes all issues for them, or if any still remain?

This fixes all my observed issues.

I think it is actually fine if we enable IndexedDB. I spent some time looking into it, and due to https://bugzilla.mozilla.org/show_bug.cgi?id=595307 and https://developer.mozilla.org/en-US/docs/Web/API/IndexedDB_API/Using_IndexedDB#Security, IndexedDB is disabled for third parties. This means we don't have to isolate it to enable it.

We will want to ensure the following:

1. IndexedDB data is cleared on New Identity. Right now, we don't do anything explicit for this, but it may get cleared automatically during cookie clearing. We should verify that.
2. IndexedDB data can't get written to disk if we're in private browsing modes.

Unfortunately, it seems as though flipping the pref doesn't actually properly enable IndexedDB. This test reports tons of errors: http://nparashuram.com/IndexedDBShim/test. Still investigating.

Trac:
Summary: TorBrowser 5.0a3 acting very odd with Twitter to Enable IndexedDB (for Twitter and other sites)

Aha, it looks as though IndexedDB is also disabled in private browsing modes: https://bugzilla.mozilla.org/show_bug.cgi?id=781982. Interestingly, it's disabled in a different way than flipping that pref, which seems good enough for us I guess?

Unfortunately, if you enable disk records/disable private browsing, New Identity does not clear IndexedDB.

Holy crap. It seems like there really is no easy way to clear indexedDB data in Firefox at all. Not through Clear Private Data, or anything else. In fact, the only code I could find that deals with IndexedDB supercookies at all is https://addons.mozilla.org/en-US/firefox/addon/mozcleaner/, and that actually removes the sqllite files and other info right off the filesystem. It also says that this is unstable (probably because the browser caches open file handles to the sqlite stores, etc).

Damn...

I am not sure if enabling indexedDB is the right move now. I am thinking it might be better to remove the pref checks in IDBFactory::CreateForWindow() and IDBFactory::CreateForMainThreadJSInternal(), so that the behavior for private browsing mode and the pref is identical (due to the checks in FactoryOp::CheckPermission()).

This should still fix twitter and ln5's bank, and is probably a better option than trying to write a bunch of code to clear indexedDB state at this point.

Trac:
Summary: Enable IndexedDB (for Twitter and other sites) to TorBrowser 5.0a3 acting very odd with Twitter due to IndexedDB

Setting 'dom.indexedDB.enabled' to true fixes all observed problems with Twitter for me. I'm not using TB in private mode, as I'm fine with the browser storing stuff on my disk currently.

Trac:
Username: Diapolo

To people who have set the pref - you are very vulnerable to SuperCookie tracking. There is no way to delete IndexedDB storage. We do not do it on New Identity, and Firefox's Clear Private Data does not do it either.

This is actually happening on a lot more sites, too. It turns out every site that uses the Modernizr feature detection library is broken with this pref flipped. Should be considered a Modernizr bug, but I bet a lot of people sites have pinned their Modernizr versions.

I have a fix for this that works for me by changing the failure mode of IndexedDB when the pref is flipped to match the failure mode for Private Browsing Mode. This patch may cause issues with e10s when we switch to ff45, and could also use another set of eyes now, too.

See https://gitweb.torproject.org/user/mikeperry/tor-browser.git/commit/?h=bug16528.

Trac:
Status: new to needs_review
Keywords: TorBrowserTeam201507 deleted, TorBrowserTeam201507R added

Trac:
Keywords: N/A deleted, GeorgKoppen201507R added
Cc: N/A to gk

Trac:
Cc: gk to gk, mcs

Tag some 5.0a4 goals.

Trac:
Keywords: tbb-5.0a deleted, tbb-5.0a4 added

Trac:
Status: needs_review to assigned
Keywords: N/A deleted, MikePerry201507 added
Owner: tbb-team to mikeperry

Trac:
Status: assigned to needs_review

Looks good to me. I am wondering why they have these checks at all if everything has to go through CheckPermission() anyway. This worries me. I tried to find that out but did not succeed. The only thing I found was https://bugzilla.mozilla.org/show_bug.cgi?id=1079355#c2 which is a more general remark.

Replying to gk:

Looks good to me. I am wondering why they have these checks at all if everything has to go through CheckPermission() anyway. This worries me. I tried to find that out but did not succeed. The only thing I found was https://bugzilla.mozilla.org/show_bug.cgi?id=1079355#c2 which is a more general remark.

Kathy and I also looked at Mike's patch and we think it is OK. But we agree with gk that it is difficult to determine why Mozilla implemented the permission checks the way they did.

Ok, I merged this for 5.0a4. At some point we should contact Mozilla about what a mess IndexedDB has become in general after the e10s rewrite, and ask them about the patch. Probably after Aug 11th, when we push to upstream our patches again.

Trac:
Resolution: N/A to fixed
Status: needs_review to closed

#16708 (moved) is a duplicate.

Trac:
Summary: TorBrowser 5.0a3 acting very odd with Twitter due to IndexedDB to TorBrowser 5.0a3 acting very odd with e.g. Twitter due to IndexedDB

Replying to mikeperry:

I have a fix for this that works for me by changing the failure mode of IndexedDB when the pref is flipped to match the failure mode for Private Browsing Mode. This patch may cause issues with e10s when we switch to ff45, and could also use another set of eyes now, too.

See https://gitweb.torproject.org/user/mikeperry/tor-browser.git/commit/?h=bug16528. Any issues with e10s? Seems the corresponding Mozilla bug is https://bugzilla.mozilla.org/show_bug.cgi?id=781982

Trac:
Severity: N/A to Normal
Reviewer: N/A to N/A
Sponsor: N/A to N/A

closed

mentioned in issue #16708 (moved)

mentioned in issue #19212 (moved)

mentioned in issue #21308 (moved)

mentioned in issue #23745 (moved)

mentioned in issue #26463 (moved)

moved to tpo/applications/tor-browser#16528 (closed)

mentioned in issue tpo/applications/tor-browser#21308 (closed)

mentioned in issue tpo/applications/tor-browser#26463 (closed)