Move TorButton code to the tor browser repository

As discussed in the[1], this is the first step to make the tor button to work on mobile.

[1] https://lists.torproject.org/pipermail/tbb-dev/2018-January/000740.html

added TBA-a2 TorBrowserTeam201811R component::applications/tor browser owner::tbb-team parent::10760 priority::very high resolution::fixed reviewer::gk reviewer::mcs reviewer::sysrqb severity::normal sponsor::8 status::closed tbb-mobile tbb-no-uplift type::defect labels

Trac:
Parent: N/A to #24855 (moved)

The code can be seen here: https://github.com/igortoliveira/tor-browser/commits/tbb-dev-desktop:

Move Tor Button source code to the browser/extensions directory https://github.com/igortoliveira/tor-browser/commit/eb2e1cec3b027647d3a873e8ce2e18102c926b7f

Integrate Tor Button in the browser build system https://github.com/igortoliveira/tor-browser/commit/d41ab6c701cbcaf00512061b05fb1e5e106a0923

Use anonymous function to keep the torbutton.js and prefereces.js scope limited to the parent function https://github.com/igortoliveira/tor-browser/commit/2c09ab0d4010c78af52a5655d865971600a215ed

Initialize startup.homepage in the startup-observer component https://github.com/igortoliveira/tor-browser/commit/b73742421735d6fea70ba4d7d550dcea22c1642c

Trac:
Status: new to needs_review

Trac:
Reviewer: N/A to gk, sysrqb, mcs

Good stuff! FWIW: we'd need a tor-browser-build patch, too, I guess making sure that Torbutton is not built anymore and not shipped as an extension anymore in Tor Browser.

Trac:
Keywords: N/A deleted, TorBrowserTeam201801R added

Created a patch removing torbutton from the tor-browser-build. ** Remove torbutton from the tor browser build ** https://github.com/igortoliveira/tor-browser-build/commit/146a721e91232e4fee31dd343d4e1bfeee9b3a3a Replying to gk:

Good stuff! FWIW: we'd need a tor-browser-build patch, too, I guess making sure that Torbutton is not built anymore and not shipped as an extension anymore in Tor Browser.

Trac:
Reviewer: gk, sysrqb, mcs to gk, sysrqb, mcs, arthuredelstein

Trac:
Cc: N/A to arthuredelstein
Reviewer: gk, sysrqb, mcs, arthuredelstein to gk, sysrqb, mcs,

I updated the code with the latest changes in Tor button, here are the new commits:

Move Tor Button source code to the browser/extensions directory https://github.com/igortoliveira/tor-browser/commit/94279657dd8f0a4c999814155633ffe5d1b1d421

Integrate Tor Button in the browser build system https://github.com/igortoliveira/tor-browser/commit/6813ba382253fd5b031bb1602cf64ce214bd274a

Use anonymous function to keep the torbutton.js and prefereces.js scope limited to the parent function https://github.com/igortoliveira/tor-browser/commit/7c77dc04844fabb2ed8ed7e4d22edc1df03b76e9

Initialize startup.homepage in the startup-observer component https://github.com/igortoliveira/tor-browser/commit/a4c6a58d287b87d8dd0736d67d7be6e87e3a30ab

Replying to igt0:

The code can be seen here: https://github.com/igortoliveira/tor-browser/commits/tbb-dev-desktop:

Move Tor Button source code to the browser/extensions directory https://github.com/igortoliveira/tor-browser/commit/eb2e1cec3b027647d3a873e8ce2e18102c926b7f

Integrate Tor Button in the browser build system https://github.com/igortoliveira/tor-browser/commit/d41ab6c701cbcaf00512061b05fb1e5e106a0923

Use anonymous function to keep the torbutton.js and prefereces.js scope limited to the parent function https://github.com/igortoliveira/tor-browser/commit/2c09ab0d4010c78af52a5655d865971600a215ed

Initialize startup.homepage in the startup-observer component https://github.com/igortoliveira/tor-browser/commit/b73742421735d6fea70ba4d7d550dcea22c1642c

Okay, I just compiled bundles with all the patches.

One thing for the tor-browser-build one: Could you add the bug number in front of your commit message like "Bug 25013:"?

For the browser patches:

1. I don't see the Torbutton icon on the toolbar button.
2. The arrow warning me that my version out-of-date is not visible either.

That's on a 64bit Linux box.

Trac:
Status: needs_review to needs_revision
Keywords: TorBrowserTeam201801R deleted, TorBrowserTeam201802 added

Oh, and please for the browser patches a "Bug 25013:" in front as well.

Replying to gk:

Okay, I just compiled bundles with all the patches.

One thing for the tor-browser-build one: Could you add the bug number in front of your commit message like "Bug 25013:"?

For the browser patches:

1. I don't see the Torbutton icon on the toolbar button.
2. The arrow warning me that my version out-of-date is not visible either.

Those two might actually not be your fault. It turns out that patch does not like the git binary diffs and seems to ignore the images in the Linux case + breaks the Windows builds. Let me try to test your patches differently tomorrow.

Update commit messages:

Tor Browser Build: ** Bug 25013: Remove torbutton from the tor browser build ** https://github.com/igortoliveira/tor-browser-build/commit/9fd151714fe5d992d9cce12e3bb59259eea154d2

Tor Browser: ** Bug 25013: Move Tor Button source code to the browser/extensions directory ** https://github.com/igortoliveira/tor-browser/commit/910da3b9edb7068a1084443e75653487997bd978

** Bug 25013: Integrate Tor Button in the browser build system ** https://github.com/igortoliveira/tor-browser/commit/e931e6ae94c5a884b6f2c8ccb96568ffff952bcd

** Bug 25013: Use anonymous function to keep the torbutton.js and preferences.js scope limited to the parent function ** https://github.com/igortoliveira/tor-browser/commit/16c360da0b7edc0218fdeac7e5300f83e9bbc0a6

** Bug 25013: Initialize startup.homepage in the startup-observer component ** https://github.com/igortoliveira/tor-browser/commit/a28341162bf034299f4b060098e62cf515716e37

Replying to gk:

Oh, and please for the browser patches a "Bug 25013:" in front as well.

Trac:
Keywords: TorBrowserTeam201802 deleted, TorBrowserTeam201802R added
Status: needs_revision to needs_review

Updated the code:

Tor Browser Build:

Bug 25013: Remove torbutton from the tor browser build
​https://github.com/igortoliveira/tor-browser-build/commit/9fd151714fe5d992d9cce12e3bb59259eea154d2

Tor Browser:

Bug 25013: Move Tor Button source code to the browser/extensions directory
​https://github.com/igortoliveira/tor-browser/commit/910da3b9edb7068a1084443e75653487997bd978

Bug 25013: Integrate Tor Button in the browser build system
​https://github.com/igortoliveira/tor-browser/commit/e931e6ae94c5a884b6f2c8ccb96568ffff952bcd

Bug 25013: Use anonymous function to keep the torbutton.js and preferences.js scope limited to the parent function
​https://github.com/igortoliveira/tor-browser/commit/16c360da0b7edc0218fdeac7e5300f83e9bbc0a6

Bug 25013: Initialize startup.homepage in the startup-observer component
​https://github.com/igortoliveira/tor-browser/commit/a28341162bf034299f4b060098e62cf515716e37

Bug 25013: Add translation tool 
https://github.com/igortoliveira/tor-browser/commit/18b32a5144aa915327cf4584873bcb25a18b625e

I am wondering what your plan is regarding the current Torbutton repository and where future development should happen (until we have all things we need in the browser and there in a non-extension-environment). Do you think we should get rid of the Torbutton repo as we currently have it and do Torbutton development directly in Tor Browser? Or what is the model you have in mind here?

Trac:
Status: needs_review to needs_information

The idea is deprecate the tor button repository(we should keep it just for historical reasons) and move the whole tor button development to tor-browser.

Replying to gk:

I am wondering what your plan is regarding the current Torbutton repository and where future development should happen (until we have all things we need in the browser and there in a non-extension-environment). Do you think we should get rid of the Torbutton repo as we currently have it and do Torbutton development directly in Tor Browser? Or what is the model you have in mind here?

Replying to igt0:

The idea is deprecate the tor button repository(we should keep it just for historical reasons) and move the whole tor button development to tor-browser.

This was my original thinking as well, but sysrqb's investigations in #25260 (moved) make me wonder if we wouldn't be better off finding a way to keep separate repositories for torbutton and tor-launcher. Perhaps we can find a way to either make them system addons ("feature" XPIs), or internal JS modules that are copied into the tor-browser.git tree at build time.

The advantage of maintaining separate torbutton and tor-launcher repositories is that we retain commit history (including the history we will accumulate for future commits). The tor-browser.git repository specifically is not so good for retaining history because we are constantly rebasing the patches to new versions of Firefox.

Replying to arthuredelstein:

Replying to igt0:

The idea is deprecate the tor button repository(we should keep it just for historical reasons) and move the whole tor button development to tor-browser.

This was my original thinking as well, but sysrqb's investigations in #25260 (moved) make me wonder if we wouldn't be better off finding a way to keep separate repositories for torbutton and tor-launcher. Perhaps we can find a way to either make them system addons ("feature" XPIs), or internal JS modules that are copied into the tor-browser.git tree at build time.

The advantage of maintaining separate torbutton and tor-launcher repositories is that we retain commit history (including the history we will accumulate for future commits). The tor-browser.git repository specifically is not so good for retaining history because we are constantly rebasing the patches to new versions of Firefox.

+1 to that.

Trac:
Status: needs_information to needs_revision
Keywords: TorBrowserTeam201802R deleted, TorBrowserTeam201802 added

FWIW: I can see us copying over files from the Torbutton repo or just package up an .xpi and put the extracted files into tor-browser.git before we tag (and use the machinery developed in some of the patches above). I just think that step 1 of the migration should not contain getting rid of the Torbutton repo. We should do this once we integrate the components we need tighter into the browser. I guess this would be a thing, at some point, for step 3 in Arthur's staged approach.

I followed the same approach in the #25260 (moved), torbutton will work as a feature extension, so we still need to change the browser adding it as a subdirectory(git submodule?). The changes in the tor button are related to the build system and making it work with the mozilla central code base(few things changed and others are deprecated).

bug 25013: Add Gecko build system infrastructure into tor button https://github.com/igortoliveira/torbutton/commit/ef153a43363ee0714f6187b545494161bea96610

**bug 25013: Migrate general.useragent.locale to intl.locale.requested ** https://github.com/igortoliveira/torbutton/commit/40682676e33570d464534adc07bf94acb36a76fe

bug 25013: Update deprecated JS code https://github.com/igortoliveira/torbutton/commit/1b50773f87465643ab8aed8104fa0c9a0fa5e0f5

bug 25013: Update XPCOM calls after changes in the interfaces https://github.com/igortoliveira/torbutton/commit/a35a746ca206706f4c5742d70bc90b649c1bfdf0

Replying to gk:

FWIW: I can see us copying over files from the Torbutton repo or just package up an .xpi and put the extracted files into tor-browser.git before we tag (and use the machinery developed in some of the patches above). I just think that step 1 of the migration should not contain getting rid of the Torbutton repo. We should do this once we integrate the components we need tighter into the browser. I guess this would be a thing, at some point, for step 3 in Arthur's staged approach.

Trac:
Status: needs_revision to needs_review

Trac:
Keywords: TorBrowserTeam201802 deleted, TorBrowserTeam201802R added

Moving our reviews to March 2018

Trac:
Keywords: TorBrowserTeam201802R deleted, TorBrowserTeam201803R added

Replying to igt0:

I followed the same approach in the #25260 (moved), torbutton will work as a feature extension, so we still need to change the browser adding it as a subdirectory(git submodule?).

Hi, Igor -- do you think torbutton could continue to be built independently inside the torbutton.git repo, and then copied as an .xpi into the Tor Browser Bundle by tor-browser-build.git? I'm wondering if can avoid adding the gecko build infrastructure.

The changes in the tor button are related to the build system and making it work with the mozilla central code base(few things changed and others are deprecated).

bug 25013: Add Gecko build system infrastructure into tor button https://github.com/igortoliveira/torbutton/commit/ef153a43363ee0714f6187b545494161bea96610

**bug 25013: Migrate general.useragent.locale to intl.locale.requested ** https://github.com/igortoliveira/torbutton/commit/40682676e33570d464534adc07bf94acb36a76fe

bug 25013: Update deprecated JS code https://github.com/igortoliveira/torbutton/commit/1b50773f87465643ab8aed8104fa0c9a0fa5e0f5

bug 25013: Update XPCOM calls after changes in the interfaces https://github.com/igortoliveira/torbutton/commit/a35a746ca206706f4c5742d70bc90b649c1bfdf0

I posted some comments on github.

Trac:
Status: needs_review to needs_revision

Trac:
Parent: #24855 (moved) to #10760 (moved)

I agree with Arthur (I have not looked at the Github commits but meant to +1 the bundling of the .xpi in the Torbutton repo and then getting that copied into the browser part later on.)

Trac:
Keywords: TorBrowserTeam201803R deleted, TorBrowserTeam201803 added

sysrqb wrote about the downsides of trying to build the .xpi outside the tor-browser repo:

https://trac.torproject.org/projects/tor/ticket/25260#comment:11

I also updated the code fixing arthur's comments:

** bug 25013: Add Gecko build system infrastructure into tor button** https://github.com/igortoliveira/torbutton/commit/86fc5398a573a43620ec3023d92207ce63def278

** bug 25013: Migrate general.useragent.locale to intl.locale.requested ** https://github.com/igortoliveira/torbutton/commit/b8443a3e75ed59283bdd52b875625534be1b96b7

** bug 25013: Update deprecated JS code ** https://github.com/igortoliveira/torbutton/commit/1b51d8afc0be10bf363b5387dbbfcd75ee021161

** bug 25013: Update XPCOM calls after changes in the interfaces ** https://github.com/igortoliveira/torbutton/commit/c68eea2c04d24e3cb47f707620365da1dbe8c7b2

** bug 25013: Initialize startup.homepage in the startup-observer component ** https://github.com/igortoliveira/torbutton/commit/cfef8182c6a5b4a294da98f0a5e71d80ce6f6472

Replying to gk:

I agree with Arthur (I have not looked at the Github commits but meant to +1 the bundling of the .xpi in the Torbutton repo and then getting that copied into the browser part later on.)

Moving our tickets to April.

Trac:
Keywords: TorBrowserTeam201803 deleted, TorBrowserTeam201804 added

Trac:
Priority: Medium to High

Move our roadmap tickets to May.

Trac:
Keywords: TorBrowserTeam201804 deleted, TorBrowserTeam201805 added

Trac:
Keywords: N/A deleted, ff60-esr added

Trac:
Keywords: N/A deleted, tbb-mobile added
Cc: arthuredelstein to arthuredelstein, dmr

Trac:
Keywords: TorBrowserTeam201805 deleted, TorBrowserTeam201806 added

Moving first batch of tickets to July 2018

Trac:
Keywords: TorBrowserTeam201806 deleted, TorBrowserTeam201807 added

Move our tickets to August.

Trac:
Keywords: TorBrowserTeam201807 deleted, TorBrowserTeam201808 added

Trac:
Keywords: ff60-esr deleted, N/A added

Moving our tickets to September 2018

Trac:
Keywords: TorBrowserTeam201808 deleted, TorBrowserTeam201809 added

Moving tickets to October

Trac:
Keywords: TorBrowserTeam201809 deleted, TorBrowserTeam201810 added

These two patches are the first steps to move tor button to firefox, it adds a new file called jar.mn that generates the chrome.manifest and update the code removing an unecessary import.

** bug 25013: Add Gecko build system infrastructure into tor button** https://github.com/igortoliveira/torbutton/commit/a86d5bbea745ff41e072a049994161eeca642e01

bug 25013: Remove non needed import, it is already available in the window object https://github.com/igortoliveira/torbutton/commit/ceba9655fff4e454bb0c4003d4d3e849abb5033c

The next step is adding torbutton as submodule in the toolkit directory and update its moz.build file.

Trac:
Status: needs_revision to needs_review

Replying to igt0:

These two patches are the first steps to move tor button to firefox, it adds a new file called jar.mn that generates the chrome.manifest and update the code removing an unecessary import.

** bug 25013: Add Gecko build system infrastructure into tor button** https://github.com/igortoliveira/torbutton/commit/a86d5bbea745ff41e072a049994161eeca642e01

bug 25013: Remove non needed import, it is already available in the window object https://github.com/igortoliveira/torbutton/commit/ceba9655fff4e454bb0c4003d4d3e849abb5033c

The next step is adding torbutton as submodule in the toolkit directory and update its moz.build file.

I guess you have that outlined in your proposal but why toolkit and not the browser directory where all the other extensions are? Let's clear review for now as other, crucial pieces are missing. The review of the parts above might depend on them being available.

Trac:
Status: needs_review to new

I would expect the browser/ directory be more about Desktop stuff. I can be totally wrong about it.

Replying to gk:

Replying to igt0:

These two patches are the first steps to move tor button to firefox, it adds a new file called jar.mn that generates the chrome.manifest and update the code removing an unecessary import.

** bug 25013: Add Gecko build system infrastructure into tor button** https://github.com/igortoliveira/torbutton/commit/a86d5bbea745ff41e072a049994161eeca642e01

bug 25013: Remove non needed import, it is already available in the window object https://github.com/igortoliveira/torbutton/commit/ceba9655fff4e454bb0c4003d4d3e849abb5033c

The next step is adding torbutton as submodule in the toolkit directory and update its moz.build file.

I guess you have that outlined in your proposal but why toolkit and not the browser directory where all the other extensions are? Let's clear review for now as other, crucial pieces are missing. The review of the parts above might depend on them being available.

Replying to igt0:

The next step is adding torbutton as submodule in the toolkit directory and update its moz.build file.

I guess you have that outlined in your proposal but why toolkit and not the browser directory where all the other extensions are? Let's clear review for now as other, crucial pieces are missing. The review of the parts above might depend on them being available. I would expect the browser/ directory be more about Desktop stuff. I can be totally wrong about it.

Yes, I think toolkit is a better location for this, too. We shouldn't add torbutton under browser/ if mobile/ needs it (which it does). It seems like toolkit/ is the directory for common browser functionality. If there is a better common directory, then I'm okay with using that, too.

I'll add review comments soon.

Trac:
Keywords: TorBrowserTeam201810 deleted, TorBrowserTeam201810R added
Status: new to needs_review

toolkit/ is fine. Some questions that came to mind while looking over the changes

1. Do we really need the bug component part of the moz.build file? That is meant for Mozilla's Bugzilla I think which we won't use anyway.

2. Why is it not enough to just have something like

% locale torbutton @AB_CD@ %locale/@AB_CD@/

but do

#if AB_CD == en_US
 locale/@AB_CD@/		(src/chrome/locale/en/*)
#else
 locale/@AB_CD@/                (src/chrome/locale/@AB_CD@/*)
#endif

and treat en-US like a special case?

3. I was wondering whether we should move the jar.mn and moz.build files into the source directory given that essentially all extensions in the Mozilla tree I have looked at have both at the same level as install.rdf.in. Did you consider that?

Trac:
Status: needs_review to needs_information

Replying to gk:

toolkit/ is fine. Some questions that came to mind while looking over the changes

1. Do we really need the bug component part of the moz.build file? That is meant for Mozilla's Bugzilla I think which we won't use anyway.

Awesome

2. Why is it not enough to just have something like

% locale torbutton @AB_CD@ %locale/@AB_CD@/

but do {{{ #if AB_CD == en_US locale/@AB_CD@/ (src/chrome/locale/en/) #else locale/@AB_CD@/ (src/chrome/locale/@AB_CD@/) #endif }}} and treat en-US like a special case?

The mozilla locale handler doesn't have an "en" language. American english is defined as en_US. So we could rename the src/chrome/locale/en folder to en_US or do this workaround. (Other extensions do the same thing)

3. I was wondering whether we should move the jar.mn and moz.build files into the source directory given that essentially all extensions in the Mozilla tree I have looked at have both at the same level as install.rdf.in. Did you consider that?

No. However I think it is a good idea. I will update the patch.

Replying to igt0:

Replying to gk:

toolkit/ is fine. Some questions that came to mind while looking over the changes

1. Do we really need the bug component part of the moz.build file? That is meant for Mozilla's Bugzilla I think which we won't use anyway.

Awesome

2. Why is it not enough to just have something like

% locale torbutton @AB_CD@ %locale/@AB_CD@/

but do {{{ #if AB_CD == en_US locale/@AB_CD@/ (src/chrome/locale/en/) #else locale/@AB_CD@/ (src/chrome/locale/@AB_CD@/) #endif }}} and treat en-US like a special case?

The mozilla locale handler doesn't have an "en" language. American english is defined as en_US. So we could rename the src/chrome/locale/en folder to en_US or do this workaround. (Other extensions do the same thing)

Are we sure "en" is the only locale in Torbutton as we ship it that is affected? That said, I think we should rather fix that issue than work around it (especially as we ship an en-US bundle and not an en one). I'll ping emmapeel for the translations related issue but hope it is not that hard to move to en-US. Otherwise we could add a symlink for now?

Trac:
Keywords: TorBrowserTeam201810R deleted, TorBrowserTeam201810 added

Okay, I filed #28261 (moved) for getting the language strings into "en-US" on the translations side.

Moving our tickets to November.

Trac:
Keywords: TorBrowserTeam201810 deleted, TorBrowserTeam201811 added

Updated code: https://github.com/igortoliveira/torbutton/tree/25013-v1

I moved the jar.mn to the src/ folder and renamed the en/ locale to en-US.

Trac:
Status: needs_information to needs_review