CI sometimes fails because it pulls the wrong image architecture

I've had a few cases where a CI job would fail because it can't execute a binary inside the container. It seems to happen because the runner pulls an i386 image instead of an amd64 one, for example:

https://gitlab.torproject.org/tpo/tpa/userdir-ldap/-/jobs/568304

Click to expand raw log

Running with gitlab-runner 16.11.0 (91a27b2a)
  on ci-runner-x86-02-main __hc2zXq, system ID: s_39a8ec4bc83a
  feature flags: FF_NETWORK_PER_BUILD:true
Preparing the "docker" executor 00:04
Using Docker executor with image debian:bullseye ...
Pulling docker image debian:bullseye ...
Using docker image sha256:ea8ad4d19589497446d3a9677d3d2303cec770aead02979b9edff18a5a1cb9e6 for debian:bullseye with digest docker.io/i386/debian@sha256:9fa77cb58cbc712cb891cde945bc882f1e70f0fe3a127693f3ac6bff9a578f4b ...
Preparing environment 00:01
Running on runner-hc2zxq-project-2253-concurrent-0 via ci-runner-x86-02...
Getting source from Git repository 00:03
Fetching changes with git depth set to 20...
Reinitialized existing Git repository in /builds/tpo/tpa/userdir-ldap/.git/
Checking out 90882c52 as detached HEAD (ref is thunderbird-pgp)...
Removing .coverage
Removing .pytest_cache/
Removing __pycache__/
Removing coverage.xml
Removing dsa-mq/
Removing report.xml
Removing test/__pycache__/
Removing userdir_ldap/__pycache__/
Skipping Git submodules setup
Executing "step_script" stage of the job script 00:02
Using docker image sha256:ea8ad4d19589497446d3a9677d3d2303cec770aead02979b9edff18a5a1cb9e6 for debian:bullseye with digest docker.io/i386/debian@sha256:9fa77cb58cbc712cb891cde945bc882f1e70f0fe3a127693f3ac6bff9a578f4b ...
$ apt-get update
Get:1 http://deb.debian.org/debian bullseye InRelease [116 kB]
Get:2 http://deb.debian.org/debian-security bullseye-security InRelease [48.4 kB]
Get:3 http://deb.debian.org/debian bullseye-updates InRelease [44.1 kB]
Err:1 http://deb.debian.org/debian bullseye InRelease
  Sub-process apt-key exited unexpectedly
Get:4 http://deb.debian.org/debian-security bullseye-security/main i386 Packages [274 kB]
Err:3 http://deb.debian.org/debian bullseye-updates InRelease
  Sub-process apt-key exited unexpectedly
Reading package lists...
W: GPG error: http://deb.debian.org/debian bullseye InRelease: Sub-process apt-key exited unexpectedly
E: The repository 'http://deb.debian.org/debian bullseye InRelease' is not signed.
W: GPG error: http://deb.debian.org/debian bullseye-updates InRelease: Sub-process apt-key exited unexpectedly
E: The repository 'http://deb.debian.org/debian bullseye-updates InRelease' is not signed.
E: Problem executing scripts APT::Update::Post-Invoke 'rm -f /var/cache/apt/archives/*.deb /var/cache/apt/archives/partial/*.deb /var/cache/apt/*.bin || true'
E: Sub-process returned an error code
Cleaning up project directory and file based variables 00:00
ERROR: Job failed: exit code 1

this is similar to #41242 (closed) except i don't mean to run this job as i386 (at least i don't think so?).. so something somewhere is telling the runner to run this with an i386 image and failing.

there are two possible solutions here:

make the i386 image work, e.g. with the physical tag (that was the solution for #41242 (closed), but i don't think it's the right one here, because this affects multiple projects and we don't want to have to do this everywhere, we want to run jobs in VM runners!)
figure out why the wrong container is pulled and fix that

/cc @lavamind