GitLab

When using remote_exec, we collect input_files in a temporary directory,
before copying them to the "remote" (for example a container). As we
don't normally modify the files inside this temporary directory, it is
safe to use hard link rather than copies of the files.

When remote_exec is not used, we don't use hard links by default, but
link_input_files can be set to 1 to use hard links.

rbm_tmp_dir is a directory inside tmp_dir that is automatically removed
when rbm exits.

We also fix an issue in urlget when used within the exec script of an
input file. In this case the input_files_id was depending on the value
of tmp_dir because it is used in urlget. We fix that be setting
getting_input_files_id when computing input_files ids, so that urlget
can avoid using tmp_dir in this case.

For simplification, we remove some unused features. Those features can
still be reimplemented externally if needed.

For example, the "rbm rpm" command can be replaced with
"rbm build --step rpm" and a definition of an rpm step in rbm.conf.

We remove the following options:
 - version_command
 - rpm
 - rpmbuild
 - srpm
 - rpmspec
 - rpm_rel
 - debian_create
 - deb
 - deb_src
 - use_pbuilder
 - debian_revision

And the following commands:
 - rpmspec
 - rpm
 - srpm
 - deb-src
 - deb
 - pkg
 - publish

This script can be used to start/stop containers during rbm builds.

In ce6b9136 we added caching of the exec template function,
however we didn't take the project into account.

Optimize the config function by reducing the number of calls to
config_p.

We also change the behavior of the function: we stop looking after the
first definition of a target. This means for example that if a target is
defined as an array in rbm.conf, and also defined as a hash in the
project's config, then the definition from rbm.conf will be ignored.

Exit with an error message if target, target_prepend or target_append
defined in input_files is not an array.

When using the build command, step gets initialized to 'build'. However
this was not done in some of the other non-build commands where the
value of step was left to rbm_init.

Instead, use the sha256file functon, which keeps the sha256sum of
files to avoid computing it twice. The function can now take a HASH as a
second argument, which can contain the key remove_cache indicating that
the sha256sum of a file should be cleared: we use this when we
find that a file has the wrong checksum and needs to be downloaded
again.

In the build_pkg call in input_files, we were setting the output_dir
option to the project's directory, causing this directory to be
incorrectly used as the output directory in some cases. But there is no
need to set output_dir at this point, so we can just remove it.

We also introduce a new feature allowing to set options for the current
project but not for the child projects listed in `input_files`. By
default the options are applied recursively to all projects. If some
options are defined under `norec/` then they are only applied on the
current project.

We use this feature in two places where we were setting the output_dir,
which removes the need to reset it in two places.

At the same time, we fix the `urlget()` call in input_files which was
incorrectly ignoring options.

See https://wiki.debian.org/ReproducibleBuilds/TimestampsInGzipHeaders

Thanks to boklm for helping with this patch.

When checking the signature on a tag, we also need to check that the tag
is really the expected tag in order to avoid rollback attacks.

Thanks to Santiago Torres-Arias and Keving Gallagher from NYU for
reporting and helping to fix this issue.

The reason we don't run |git checkout --detach| in all cases is that it
fails when we cloned a repository without a master branch. To avoid
running it in this case, we were checking if HEAD is pointing to a
branch which has a corresponding file in the .git directory.

However it seems there are other cases where HEAD is pointing to a
branch which does not have a corresponding file in the .git directory.
Instead we now check that |git rev-parse --verify HEAD| does not return
an error, in which case we assume that |git checkout --detach| will not
produce an error too.

Set DEBIAN_FRONTEND=noninteractive when using apt-get in install_package.
Also use the -q (quiet) argument.

We replace a call to:
  git submodule update --init
with calls to:
  git submodule init
  git submodule sync
  git submodule update

The call to `git submodule sync` is solving the issue that could occur
when a submodule URL is changing.

To create an archive containing all submodules, we were creating a
temporary archive of each submodule before appending them to the main
archive. We were using the submodule path in the temporary archive
filename, which was failing if the submodule is in a subdirectory.
To avoid that, we are removing the path from the temporary archive's
filename and directly appending it to the main archive.

By default gpg will from time to time update its trust database. When
this happens it will also modify the keyring files to add some trust
information. To avoid this we add the --no-auto-check-trustdb option.

As we don't use the Web of Trust when we use a keyring file, we also
disable it using `--trust-model always`.

Print stderr from the git/hg command if the checkout failed.