Fix build toolchain creating root-owned files on Linux #342
Conversation
flaki
force-pushed
the
flaki/175-fix-root
branch
from
2c40513 to
003839f
Compare
flaki
changed the title
flaki
force-pushed
the
flaki/175-fix-root
branch
from
003839f to
a51f0ae
Compare
|
Updated to track |
flaki
force-pushed
the
flaki/175-fix-root
branch
from
a51f0ae to
06f1dff
Compare
flaki
requested review from
cohix,
javorszky,
arbourd,
danielledeleo and
ospencer
|
The latest update:
About that last part: We are calling the Docker toolchain as a limited user, quite possibly a user that does not even exist inside the container (thus,
|
Co-authored-by: Oscar Spencer <[email protected]>
| uid := os.Getuid() | ||
| gid := os.Getgid() | ||
|
|
||
| toolchainCmd := fmt.Sprintf("docker run --rm --mount type=bind,source=%s,target=/usr/src/runnable -u %d:%d --env HOME=/tmp %s subo build %s --native --langs %s", b.Context.MountPath, uid, gid, img, b.Context.RelDockerPath, lang) |
There was a problem hiding this comment.
This is great and obvious in hindsight!
|
|
||
| FROM node:16-buster-slim | ||
| WORKDIR /root/runnable | ||
| WORKDIR /usr/src/runnable |
There was a problem hiding this comment.
Why was this path chosen? /usr/* paths are usually root owned as well and I don't see a chown anywhere for it. Does all of the writing happen in the /tmp dir set as $HOME in the Docker invocation?
There was a problem hiding this comment.
I...honestly don't remember where /usr/src came from, I definitely remember /root being an issue though.
The /tmp as HOME change came later for the Go/Swift images and I don't remember that being needed for other images, but I think this might vary depending on what's the UID/GID of the invoking user.
| docker build . -f builder/docker/assemblyscript/Dockerfile -t suborbital/builder-as:dev | ||
| docker build . -f builder/docker/grain/Dockerfile --platform linux/amd64 -t suborbital/builder-gr:dev | ||
| docker build . -f builder/docker/javascript/Dockerfile -t suborbital/builder-js:dev | ||
| docker build . -f builder/docker/rust/Dockerfile -t suborbital/builder-rs:dev | ||
| docker build . -f builder/docker/swift/Dockerfile -t suborbital/builder-swift:dev | ||
| docker build . -f builder/docker/tinygo/Dockerfile --platform linux/amd64 --build-arg TARGETARCH=amd64 -t suborbital/builder-tinygo:dev | ||
| docker build . -f builder/docker/wat/Dockerfile -t suborbital/builder-wat:dev |
There was a problem hiding this comment.
We don't have an explicit dev target for local builds (which is weird) but these can be replaced with calls to make like this:
make ver=dev builder/docker/tinygoThere was a problem hiding this comment.
Cool, I'll change it around!
Co-authored-by: Jagger De Leo <[email protected]>
Fixes #175
Discord thread
As noted in that issue, on Linux, Docker is run as
rootand this is how the current runnable is mounted into the Docker filesystem. This results in root-owned build artifacts created in the user's directory that, among other things, cannot be deleted (withoutsudo).This does not seem to happen on most other platforms due to specific ways how Docker mounts folders on those platforms.
This fix explores the solution of using
docker run -uto run the invoked toolchain with the current user (os.Getuid).Running as a limited user breaks at least the Rust builder which needed to be updated slightly, and may break other builders, I will do some testing around those and will include any fixes in here, will keep this as a draft PR until then.
Rust builder breakage
For posterity, the reason for this breakage is that while the official upstream Rust builder makes relevant directories (such as the one used by cargo for the registry index) world-writable, our builder image (which still runs as
rootat this point) initializes the index asroot, and the files and folders created by it need to bechmod-ded to allow writing by anyone, otherwise the limited user builds break when the command tries to write into these directories.