Unverified Commit 219a86f0 authored by Matthew Finkel's avatar Matthew Finkel
Browse files

Merge remote-tracking branch 'gkgl/bug_33932_v6'

parents e370ae7a 061b9e7b
......@@ -3,48 +3,27 @@ If additional Android dependencies are required by the project's build, then
the Gradle build will fail due to missing dependencies. To find out what the
missing dependencies are take the following steps.
For tor-onion-proxy-library and tor-android-service, replace the following line
in the respective project build file:
$GRADLE_HOME/gradle-4.10.2/bin/gradle --offline --no-daemon -P androidplugin=3.1.0 -Dmaven.repo.local=$gradle_repo assembleRelease -x lint
with
$GRADLE_HOME/gradle-4.10.2/bin/gradle --debug --no-daemon -P androidplugin=3.1.0 assembleRelease -x lint
When calling gradle in the project's build script replace the `--offline` flag
with `--debug` and remove any `-Dmaven.repo.local` arguments.
For the firefox project, comment out the following line in the project's build file:
export GRADLE_MAVEN_REPOSITORIES="file://$rootdir/[% c('input_files_by_name/gradle-dependencies') %]"
export GRADLE_MAVEN_REPOSITORIES="file://$gradle_repo"
Also modify the gradle flags to include the debug option so the download logs will show up:
export GRADLE_FLAGS="--no-daemon --debug"
then allow network access during the build by setting
var/container/disable_network/build to 0 in rbm.conf, and rerun the build.
Then allow network access during the build by setting
`var/container/disable_network/build` to `0` in rbm.conf, and rerun the build.
Dependent artifacts will show up as downloads in the logs. You can pull out
these dependencies into a list with the following command (replacing
"firefox-android-armv7.log" with the build log file name of the actual project):
these dependencies into a list by passing the log file to the gradle dependency
list script in the tools directory:
`cat logs/firefox-android-armv7.log | grep "Performing HTTP" | grep -o "https://.*" | sort | uniq > download-attempts.txt`
`./gen_gradle_deps_file.sh /path/to/log/file`
The download-attempts.txt file contains all the attempted downloads, so we need to find out which ones failed
`cat logs/firefox-android-armv7.log | grep "Resource missing" | grep -o "https:.*[^]]" | sort | uniq > download-fails.txt`
Now take the intersection. This removes failures from attempts, leaving just successful downloads
`sort download-attempts.txt download-fails.txt | uniq -u | rev | sort -t/ -u -k1,4 | rev | sort > download-urls.txt`
You will then need to add the new dependency URLs and SHA-256 values into the
projects/$project/gradle-dependencies-list.txt file. The format of this file is
pipe delimited
sha256sum | url
Finally, in the project's config file increment the
var/gradle_dependencies_version and make sure to restore the project's build
Copy the resulting `gradle-dependencies-list.txt` over the one in the respective
project. Then, in the project's config file, increment the
`var/gradle_dependencies_version` and make sure to restore the project's build
file back to original.
It may also be the case that you wish to clean up old versions of the artifacts.
For this you will need to run the build with a
gradle-dependencies-list.txt file containing only the headers. Make sure to also
comment the GRADLE_MAVEN_REPOSITORIES line from the project's build file. You
can now proceed to reconstruct the list as given above.
#!/bin/sh
# Copyright (c) 2020, The Tor Project, Inc.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met:
#
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
#
# * Redistributions in binary form must reproduce the above
# copyright notice, this list of conditions and the following disclaimer
# in the documentation and/or other materials provided with the
# distribution.
#
# * Neither the names of the copyright owners nor the names of its
# contributors may be used to endorse or promote products derived from
# this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
# Usage:
# 1) Point to a log file with all the dependency download attempts (for its
# generation see: projects/common/how-to-create-gradle-dependencies-list.txt)
# 2) Double-check that you get the same SHA-256 sums when downloaded from a
# different network location. E.g. by using `torsocks` with this script after
# having made a copy of `gradle-dependencies-list.txt` from 1) and comparing
# the two .txt files.
log="$1"
# Step 1: Extract all the download attempts out of the log file, ignore the ones
# for maven-metadata.xml files. We don't need those.
cat $log | grep "Performing HTTP" | grep -o "https://.*" | \
grep -v "maven-metadata.xml" | sort | uniq > dl-attempts
# Step 2: Fetch all the dependencies and calculate the SHA-256 sum
while read line
do
wget -U "" $line
fn=$(basename "$line")
sha256=`sha256sum $fn | cut -d ' ' -f 1`
echo "$sha256 | $line" >> deps
rm $fn
done < dl-attempts
# Step 3: Add the header at the beginning of the final dependency file.
echo "# On how to update dependencies see projects/common/how-to-create-gradle\
-dependencies-list.txt" > gradle-dependencies-list.txt
echo "# Don't forget to update var/gradle_dependencies_version when modifying \
this file" >> gradle-dependencies-list.txt
echo "sha256sum | url" >> gradle-dependencies-list.txt
# Step 4: Keep only successfully downloaded artifacts, remove duplicates, and
# sort based on download URL.
grep ^[a-f0-9] deps | rev | sort -t/ -u -k1,4 | rev | \
sort -k 3 >> gradle-dependencies-list.txt
# Step 5: Clean up
rm dl-attempts
rm deps
exit 0
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment