Update from 6.5a2 to 6.5a3 on OSX breaks Tor Browser

added TorBrowserTeam201610R component::applications/tor browser owner::tbb-team priority::very high resolution::fixed severity::major status::closed type::defect labels

Trac:
Description: After updating to 6.5a3, OSX returns an error when trying to open the Tor Browser: "The application “TorBrowser” can’t be opened."

Testing from terminal:

$ open /Applications/TorBrowser.app
LSOpenURLsWithRole() failed with error -10810 for the file /Applications/TorBrowser.app.

OXS version: 10.11.6

I was able to reproduce the issue with a clean install of 6.5a2 in a different directory.

to

After updating to 6.5a3, OSX returns an error when trying to open the Tor Browser: The application “TorBrowser” can’t be opened.

Testing from terminal:

$ open /Applications/TorBrowser.app
LSOpenURLsWithRole() failed with error -10810 for the file /Applications/TorBrowser.app.

OXS version: 10.11.6

I was able to reproduce the issue with a clean install of 6.5a2 in a different directory.

I tested that on my old 10.6 system and got

posix_spawn("/path/to/TorBrowser.app/Contents/MacOS/firefox", ...):
Permission denied

Not sure whether both are related.

Updating from 6.0.4 to 6.0.5 is working fine and starting a vanilla 6.5a3 is working as well.

#20211 (moved) is a duplicate.

Trac:
Priority: Medium to Very High

Perhaps in the mean time, we should let people know this upgrade might break their Tor Browsers and have them wait till the next one comes out or ask them to manually download 6.5a3.

I was able to successfully update and open an older en-US 6.5a2 installation to 6.5a3 on one OSX 10.11.6 system.

On another system, I tried to update a newly installed es-US 6.5a2 browser but it did not go well. The first thing I noticed is that the incremental MAR failed to apply. I will have to investigate more tomorrow, but last-update.log contains:

EXECUTE PATCH Contents/MacOS/updater.app/Contents/MacOS/updater
LoadSourceFile: destination file size 106848 does not match expected size 97000
LoadSourceFile failed
### execution failed

The complete MAR was applied successfully, but then the app will not open. It looks like the mode is wrong on some files, e.g., TorBrowser.app/Contents/MacOS/firefox has mode 644 instead of 755. I am out of time for today, but maybe (some of?) the MAR files are wrong.

Okay, I learned a little bit more. After I did this the browser comes much closer to starting up:

cd TorBrowser.app
chmod +x Contents/MacOS/firefox
chmod +x Contents/MacOS/Tor/tor.real
chmod +x Contents/Resources/TorBrowser/Tor/tor

Unfortunately, I then encountered #18753 (moved) for the control port socket because my browser is installed in /Applications, which means the control socket path contains a space:

~/Library/Application Support/TorBrowser-Data/Tor/control.socket

(in hindsight, I now realize that #18753 (moved) is going to affect a lot of people).

Finally, after I created a /tmp/sockets directory with mode 700 and added an appropriate extensions.torlauncher.control_socket_path preference, my 6.5a3 browser starts and works.

We definitely need to find a way to test updates before we release :(

#20215 (moved) is a duplicate. Stack Exchange link from there: Tor and macOS Sierra Screenshot:

The file permission/mode problem may be related to #19410 (moved) (but I have no proof). Because of #18753 (moved), we should toggle extensions.torlauncher.control_port_use_socket to false for now. I do not yet have any theories about the incremental MAR file problem.

Trac:
Cc: mcs, brade to mcs, brade, boklm

Replying to mcs:

I do not yet have any theories about the incremental MAR file problem.

This is still happening as #19410 (moved) is only partly fixed: the 6.5a2 .mar files did not contain code-signed parts yet. The 6.5a3 ones are the first that do. Thus, the incremental ones are still wrong. This should go away as soon as all .mar files involved in generating the incremental ones do contain the necessary code-signed bits.

Replying to gk:

This is still happening as #19410 (moved) is only partly fixed: the 6.5a2 .mar files did not contain code-signed parts yet. The 6.5a3 ones are the first that do. Thus, the incremental ones are still wrong. This should go away as soon as all .mar files involved in generating the incremental ones do contain the necessary code-signed bits.

That makes sense. Can we remove the flawed incremental MAR files and stop offering them to 6.5a2 users? I guess we did that for the stable channel; maybe it is not worth the effort for alpha at this point.

Replying to mcs:

The file permission/mode problem may be related to #19410 (moved) (but I have no proof).

Kathy and I have confirmed that file mode is lost inside dmg2mar during the 7z extraction step. Unless there is a 7z option to preserve this, it is probably a bug in 7z. In the short run, maybe we can:

use the mar tool to extract each complete mar file
manually fix file and dir modes to match those used inside the dmg
use the mar tool to re-create the mar
push the fixed mar files to dist.tpo

Replying to mcs:

Replying to mcs:

The file permission/mode problem may be related to #19410 (moved) (but I have no proof).

Kathy and I have confirmed that file mode is lost inside dmg2mar during the 7z extraction step. Unless there is a 7z option to preserve this, it is probably a bug in 7z. In the short run, maybe we can:

use the mar tool to extract each complete mar file

manually fix file and dir modes to match those used inside the dmg

use the mar tool to re-create the mar

push the fixed mar files to dist.tpo

I am not sure about that. At least I won't have time for that tomorrow. Not sure about Monday. boklm or anybody else: if you feel this is worthwhile, go ahead. Otherwise I'd say this is a nasty alpha bug which one can avoid by downloading a fresh new .dmg. Or following the advice on the blog which links to comment:5.

Replying to mcs:

Unfortunately, I then encountered #18753 (moved) for the control port socket because my browser is installed in /Applications, which means the control socket path contains a space: {{{ ~/Library/Application Support/TorBrowser-Data/Tor/control.socket }}} (in hindsight, I now realize that #18753 (moved) is going to affect a lot of people).

This makes a lot of sense! I didn't get this error because I installed my fresh copy in ~/Downloads directory, hence no space in path.

On MacOS 10.6.8, I can confirm that the space in the default path to TorBrowser-Data/ does indeed seem to be at least part of the cause. I installed a clean copy of TorBrowser 6.5a3 to the Desktop and ran it from there, and it starts normally. Installing it in /Applications causes the "Could not connect to Tor control port" error described above.

Trac:
Username: heywoodj123@yahoo.com

Trac:
Keywords: N/A deleted, TorBrowserTeam201609 added

I am also able to reproduce the problem in Tor Browser 6.0.5 when a separate alpha series is upgraded to 6.5a3, or is freshly installed, on the same machine.

Updating an alpha series Tor Browser from 6.5a2 to 6.5a3 causes a previously updated stable series Tor Browser 6.0.5, when launched subsequently on the same Mac OS X/macOS system, to show the same control port failure symptom. Ditto a freshly installed 6.0.5. This behavior was noted on 3 separate Mac OS X systems (2x 10.11.6, 1x 10.12).

On the affected machines, the prefs.js edit to add 'user_pref("extensions.torlauncher.control_port_use_socket", false);' allows 6.0.5 and 6.5a3 versions to both connect again.

This suggests the cause is coupled to the changes made in shared resources at "~/Library/Application Support/TorBrowser-Data/Browser/..." by the 6.5a3 upgrade.

Replying to rdump:

I am also able to reproduce the problem in Tor Browser 6.0.5 when a separate alpha series is upgraded to 6.5a3, or is freshly installed, on the same machine.

Updating an alpha series Tor Browser from 6.5a2 to 6.5a3 causes a previously updated stable series Tor Browser 6.0.5, when launched subsequently on the same Mac OS X/macOS system, to show the same control port failure symptom. Ditto a freshly installed 6.0.5. This behavior was noted on 3 separate Mac OS X systems (2x 10.11.6, 1x 10.12).

Thank you for adding to this ticket. I assume you are running both the 6.0.5 and 6.5a3 browsers from /Applications? Please do not do that for both the stable and alpha releases, because the user profile will be shared (which is not necessarily a good thing). What is happening in this specific case is that when you open TB 6.5a3, the versions of Torbutton and Tor Launcher that ship with the alpha get installed in your browser profile under ~/Library/Application Support/TorBrowser-Data. Unfortunately, later when you open TB 6.0.5, the same Torbutton and Tor Launcher are used. And now your 6.0.5 is broken because it is running with alpha extensions.

I recommend that you keep the alpha in a folder outside of /Applications so it will always have its own TorBrowser-Data directory.

Replying to mcs:

Replying to mcs:

The file permission/mode problem may be related to #19410 (moved) (but I have no proof).

Kathy and I have confirmed that file mode is lost inside dmg2mar during the 7z extraction step. Unless there is a 7z option to preserve this, it is probably a bug in 7z. In the short run, maybe we can:

As part of the OS X signing process, we are creating *-signed.tar.bz2 files containg the code signed OS X bundles, and we have a script that use them to create the .dmg files. Rather than using the dmg files (and 7z to extract them) as the source to create the code signed mar files, we should maybe use the *-signed.tar.bz2 files instead.

Replying to boklm:

Replying to mcs:

Replying to mcs:

The file permission/mode problem may be related to #19410 (moved) (but I have no proof).

Kathy and I have confirmed that file mode is lost inside dmg2mar during the 7z extraction step. Unless there is a 7z option to preserve this, it is probably a bug in 7z. In the short run, maybe we can:

As part of the OS X signing process, we are creating *-signed.tar.bz2 files containg the code signed OS X bundles, and we have a script that use them to create the .dmg files. Rather than using the dmg files (and 7z to extract them) as the source to create the code signed mar files, we should maybe use the *-signed.tar.bz2 files instead.

Hm. We need the signed .dmg files anyway and just copying all the .bz2 bundles over as well to solve the permission problem seems to be a bit of over-engineering to me. What speaks against adjusting the permissions with the dmg2mars script (if we can't solve this problem with 7z)?

Replying to gk:

Hm. We need the signed .dmg files anyway and just copying all the .bz2 bundles over as well to solve the permission problem seems to be a bit of over-engineering to me. What speaks against adjusting the permissions with the dmg2mars script (if we can't solve this problem with 7z)?

It seems error prone (fragile) for us to adjust the permissions. How do we know what permissions should be set?

Replying to brade:

Replying to gk:

Hm. We need the signed .dmg files anyway and just copying all the .bz2 bundles over as well to solve the permission problem seems to be a bit of over-engineering to me. What speaks against adjusting the permissions with the dmg2mars script (if we can't solve this problem with 7z)?

It seems error prone (fragile) for us to adjust the permissions. How do we know what permissions should be set?

We set them already before creating the .dmg file with the ddmg.sh script, no?

Replying to gk:

Hm. We need the signed .dmg files anyway and just copying all the .bz2 bundles over as well to solve the permission problem seems to be a bit of over-engineering to me. What speaks against adjusting the permissions with the dmg2mars script (if we can't solve this problem with 7z)?

Ah yes, fixing that issue by setting the permissions like ddmg.sh is doing should be simple. I think we can do that.

If I understand the process we have correctly, we are creating *-signed.tar.bz2 files on the signing machine, we copy them over an other machine where we run a script to create the dmg files from the *-signed.tar.bz2 files, and we then use dmg2mar to convert those dmg files to mar files. I was thinking that the creation of the dmg and the mar files could be done at the same time, but maybe that should be a separate ticket.

Replying to boklm:

Replying to gk:

Hm. We need the signed .dmg files anyway and just copying all the .bz2 bundles over as well to solve the permission problem seems to be a bit of over-engineering to me. What speaks against adjusting the permissions with the dmg2mars script (if we can't solve this problem with 7z)?

Ah yes, fixing that issue by setting the permissions like ddmg.sh is doing should be simple. I think we can do that.

If I understand the process we have correctly, we are creating *-signed.tar.bz2 files on the signing machine, we copy them over an other machine where we run a script to create the dmg files from the *-signed.tar.bz2 files, and we then use dmg2mar to convert those dmg files to mar files. I was thinking that the creation of the dmg and the mar files could be done at the same time, but maybe that should be a separate ticket.

Aha! Yes, this should be a separate ticket. The .bz2 thing is only a stopgap and is outside of the gitian machinery currently (and I hope we get to a point where we don't need it or a similar stopgap anymore but that is going to take a bit of time).

Replying to gk:

We set them already before creating the .dmg file with the ddmg.sh script, no?

I might be overlooking something, but it looks like ddmg.sh relies on some of the mode bits being correct in order to determine which files should have mode 0755, etc. Since the exec bits are lost during the 7z extraction process, I don't think ddmg.sh will work as-is (but I have not tried it).

Replying to mcs:

Replying to gk:

We set them already before creating the .dmg file with the ddmg.sh script, no?

I might be overlooking something, but it looks like ddmg.sh relies on some of the mode bits being correct in order to determine which files should have mode 0755, etc. Since the exec bits are lost during the 7z extraction process, I don't think ddmg.sh will work as-is (but I have not tried it).

I think you are right, ddmg.sh is using find $@ -executable to find which files to change.

I can see two options to fix this permissions problem:

generate the new mar files using the *-signed.tar.bz2 files instead of the .dmg files
generate them from the .dmg files, and extract the old mar file to replicate the file permissions from the old mar file to the new one

Replying to boklm:

Replying to mcs:

Replying to gk:

We set them already before creating the .dmg file with the ddmg.sh script, no?

I might be overlooking something, but it looks like ddmg.sh relies on some of the mode bits being correct in order to determine which files should have mode 0755, etc. Since the exec bits are lost during the 7z extraction process, I don't think ddmg.sh will work as-is (but I have not tried it).

I think you are right, ddmg.sh is using find $@ -executable to find which files to change.

I can see two options to fix this permissions problem:

generate the new mar files using the *-signed.tar.bz2 files instead of the .dmg files

generate them from the .dmg files, and extract the old mar file to replicate the file permissions from the old mar file to the new one

So, I really like to avoid roping the .bz2 files in as this adds an additional kludge to this signing nightmare. Two additional options:

maybe we can look at Mozilla code checking how the correct permission are set in the first place and replicate that
fix 7z as this seems clearly to be a bug that deserves to get fixed upstream

My hope is that fixing the permission problem in 7z is not that hard which is why I would start with that one first. Otherwise taking the option that consumes the least amount of time to implement and test seems to be a good heuristic. If it turns out to include the .bz2 files I guess I'd bite that bullet.

Moving tickets to October.

Trac:
Keywords: TorBrowserTeam201609 deleted, TorBrowserTeam201610 added

I have looked at why 7z doesn't set the correct permissions. It seems that it is because it doesn't read the RockRidge extensions on the iso image, containing the permissions (and the symlink infos). It looks like some support for this has been added to 7zip 16.04, but the latest p7zip version (the port of 7zip to Unix which we are using) is still based on 7zip 16.02. I patched p7zip with the changes from 7zip adding support for the RockRidge extensions, but it still does not set the permissions correctly. I started debugging this but did not find yet what is happening.

Trac:
Cc: mcs, brade, boklm to mcs, brade, boklm, arthuredelstein

It looks like fixing 7z to handle file permissions on our dmg files will require a lot of debugging. As a workaround to avoid this 7z bug, the branch bug_20210 in my repository has a patch that extract the old mar file to copy the permissions to the new mar file: https://gitweb.torproject.org/user/boklm/tor-browser-bundle.git/commit/?h=bug_20210

Trac:
Status: new to needs_review
Keywords: TorBrowserTeam201610 deleted, TorBrowserTeam201610R added

After some trouble with testing (if the permissions are already wrong on the alpha .mar file it won't help taking that .mar file for trying to fix them, right? ;) ) I got it working and the patch looks good to me. Fixed on master with commit 97acdebccac65377317068cdbfae6def9bb67309.

Trac:
Status: needs_review to closed
Resolution: N/A to fixed

closed

mentioned in issue #20211 (moved)

mentioned in issue #20215 (moved)

mentioned in issue #20250 (moved)

mentioned in issue #20627 (moved)

moved to tpo/applications/tor-browser#20210 (closed)

Update from 6.5a2 to 6.5a3 on OSX breaks Tor Browser

Child items 0

Activity