Skip to content

Consider mounting out as a bind

In tpo/tpa/team#40964 (closed) micah suggested we switch to bind mounts to avoid copying.

I agree that it'd be a great idea, especially since almost all of the files we copy are archives we then just extract and don't use.

At least theoretically, I don't have numbers at the moment about the overhead that copies create, but I'd love if we instrumented RBM to profile the steps that take a long time. E.g., starting to build firefox-android on my machine takes 8 mins from the creation of the log to the start of the build script. In tb-build-05, the same takes 11 minutes (I don't expect file copies to be the main cause of this overhead, though). (But these are exceptions, most of the projects are below 30 seconds; see #40061 (closed)).

I see a couple of concerns, but we could easily solve them in my opinion:

  1. we want to keep visible only a limited set of inputs
    • We could share a temporary directory inside as out on which we hardlink the actual inputs. AFAIK, hardlinking them to the directory of the container doesn't work in some environments, e.g., if we use a ramdisk (like we do in tb-build-06), but binding volume should work.
  2. it might be more difficult to find the exact file
    • maybe we could change c("input_files_by_name/...") to be absolute paths.
    • if we prepare a single directory outside the container and expose that, finding files should be easy.
Edited by Pier Angelo Vendrame
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information