Create/provide access to a VM for running Tor Browser integration tests

acat recently finished fixing our Tor Browser integration tests so that they are running properly for Linux bundles again. We contacted boklm but the old VM he was using is not available anymore. So, we either need a new one or just use an existing one as the requirements for running our tests are not particularly high. What we need:

1. acat, sysrqb, and gk would need access to that VM to be able to update tests, debug issues etc.

2. Our tests are in our tor-browser-bundle-testsuite repository. So, we would need to clone that one and then someone with sudo powers need to install packages as specified in install-deps

3. As to RAM and CPU requirements I think essentially anything might do it

4. OS should be Debian stable, 64bit.

5. Hard disk requirements are low, too. 5GiB should be plenty in particular as we would start with running the tests for en-US bundles only in the beginning anyway and the test reports are not large.

6. The system should be able to send mail to acat, sysrqb, and gk for test result notifications.

cc @acat, @sysrqb, @boklm

moved from gitlab#82 (moved)

assigned to @anarcat

added Backlog label

added Next label and removed Backlog label

i'm happy to announce we have capacity again. some questions:

1. acat, sysrqb, and gk would need access to that VM to be able to update tests, debug issues etc.

which kind of access? shell? sudo?

2. Our tests are in our tor-browser-bundle-testsuite repository. So, we would need to clone that one and then someone with sudo powers need to install packages as specified in install-deps

which sudo powers? to root? can't we just install the deps for you?

3. As to RAM and CPU requirements I think essentially anything might do it
4. OS should be Debian stable, 64bit.

our basic VM is Debian stable 64bits, 8GB, 2CPUs, is that okay?

5. Hard disk requirements are low, too. 5GiB should be plenty in particular as we would start with running the tests for en-US bundles only in the beginning anyway and the test reports are not large.

awesome.

6. The system should be able to send mail to acat, sysrqb, and gk for test result notifications.

also standard.

i hope to be able to build you a machine next week. what should it be called? see also https://gitlab.torproject.org/tpo/tpa/team/-/wikis/doc/naming-scheme

anarcat commented:

i'm happy to announce we have capacity again. some questions:

1. acat, sysrqb, and gk would need access to that VM to be able to update tests, debug issues etc.

which kind of access? shell? sudo?

Only shell and sudo to a common user. I'll suggest testsuite as the common user name, but tester may be a better name.

2. Our tests are in our tor-browser-bundle-testsuite repository. So, we would need to clone that one and then someone with sudo powers need to install packages as specified in install-deps

which sudo powers? to root? can't we just install the deps for you?

Yes, please install the deps for us :)

We can clone the git repo, and then someone from tpa can install the deps:

   apt-get install -y python-virtualenv
   apt-get install -y strace
   # - Cwd : in core since perl 5
   # - Digest::SHA : in core since perl v5.9.3
   # - File::Path : in core since perl 5.001
   # - File::Spec : in core since perl 5.00405
   # - File::Temp : in core since perl v5.6.1
   # - Getopt::Long : in core since perl 5
   # - FindBin : in core since perl 5.00307
   # - IO::Socket::INET in core since perl v5.6.0
   # - Storable : in core since perl v5.7.3
   apt-get install -y libdata-dump-perl libfile-slurp-perl \
            libio-captureoutput-perl perlmagick libjson-perl \
            libwww-perl liblwp-protocol-https-perl libtemplate-perl \
            libyaml-syck-perl libdatetime-perl \
            libemail-sender-perl libemail-simple-perl libfile-type-perl \
            libipc-run-perl libxml-libxml-perl
   apt-get install -y nmap curl

https://gitweb.torproject.org/tor-browser-bundle-testsuite.git/tree/install-deps#n45

Plus the X dependencies:

fluxbox xserver-xorg-video-dummy xserver-xorg-input-void libgtk2.0-0 iceweasel

https://gitweb.torproject.org/tor-browser-bundle-testsuite.git/tree/install-deps#n105

(I hope that includes all needed dependencies)

3. As to RAM and CPU requirements I think essentially anything might do it
4. OS should be Debian stable, 64bit.

our basic VM is Debian stable 64bits, 8GB, 2CPUs, is that okay?

Yes, that sounds great :)

5. Hard disk requirements are low, too. 5GiB should be plenty in particular as we would start with running the tests for en-US bundles only in the beginning anyway and the test reports are not large.

awesome.

6. The system should be able to send mail to acat, sysrqb, and gk for test result notifications.

also standard.

i hope to be able to build you a machine next week. what should it be called? see also https://gitlab.torproject.org/tpo/tpa/team/-/wikis/doc/naming-scheme

I'll offer tekesicola, but I'm open to other suggestions :)

Thanks!

I'll offer tekesicola, but I'm open to other suggestions :)

we discussed this on IRC and agreed on tb-tester-01 instead. i updated the naming docs to better reflect current policy (which is to avoid onions, unfortunately). (yes, i am a soul-less documenter. ;))

everything else here is go, i'll see if i can create the VM by EOD.

thanks!

added Doing label and removed Next label

er. i should also mention a HUGE caveat: you're going to be the first virtual machine hosted at our new (rather untested) Cymru cluster. it's a Ganeti cluster, like the one we have at Hetzner, so things should work the same, but as I said it's untested, so it might not behave the same and could crash and burn. it will have backups like everything else, but I thought I should mention it.

i figure it's actually a pretty good machine to test the cluster with since, well, it's going to run tests. ;) but let me know if that's a problem and i'll see if i can find a workaround.

the VM has been created:

root@tb-tester-01:~# uptime ; free -h ; grep -c ^proc /proc/cpuinfo  ; df -h -t ext4
 16:32:40 up 12 min,  1 user,  load average: 0.03, 0.05, 0.01
              total        used        free      shared  buff/cache   available
Mem:          7.8Gi       106Mi       7.5Gi       8.0Mi       213Mi       7.5Gi
Swap:         2.0Gi          0B       2.0Gi
2
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda1       9.8G  2.4G  6.9G  26% /
/dev/sdc         20G   45M   19G   1% /srv

... and the dependencies installed. the role/group is called tb-tester to fit with the server name.

@acat, @sysrqb and @gk have sudo access, as they are part of the @tb-tester group.

everything should be good here, please do reopen the ticket if i missed anything (or open a new one).

closed

@anarcat There are some missing deps (at least python-virtualenv and libpath-tiny-perl, probably others). I could not find a way to run a command as root (it seems sudo only lets me run commands as the tb-tester user).

reopened

python-virtualenv is definitely installed:

root@tb-tester-01:~# dpkg -l python-virtualenv
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name              Version      Architecture Description
+++-=================-============-============-==================================
ii  python-virtualenv 15.1.0+ds-2  all          Python virtual environment creator

... or did you mean python3-virtualenv?

as for libpath-tiny-perl, it's the first time I hear about it. :) it's not listed in the original list you provided in this ticket, was I supposed to go through install.sh and figure out the deps myself? it doesn't look like it's listed there either... i installed it, in any case...

reopen with specific packages you need, and i'll be happy to oblige. :)

closed

after talking with acat directly on IRC, ended up installing virtualenv and a few more dependencies here, manually on the host but also committed to puppet.

it seems we're good on deps now.