-
Notifications
You must be signed in to change notification settings - Fork 11
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use pre-commit hooks in tox configuration #2057
base: master
Are you sure you want to change the base?
Conversation
IMHO nobody should run dev tools as root -- should be mapped to the real UID via |
Should the difficulties around running linting inside docker be documented in |
when do |
Nobody is running dev tools as root 🙂. Docker containers, by default, run as root inside the container (even when launched by a non-root user on the host system). Your example is using Singularity, which was designed in part to work with non-root users by default.
Tox needs to run inside the container precisely when the dev wants it to. For example, I often run the command Resonant applications provides this container as a convenience. Adopting pre-commit in dandi-archive caused some of the "rev matching" problems to surface, so I'm looking into how to fix it to provide a generally better experience across Resonant apps. In any case, I am working on a way to build the container so that it runs as the same user as the dev who built it. If that works, then it will provide parity between host OS and Docker container. Stay tuned for updates. |
FWIW -- checked that it works: ❯ docker compose run --rm --user "$(id -u)" django touch 123
WARN[0000] /home/yoh/proj/dandi/dandi-archive/docker-compose.yml: the attribute `version` is obsolete, it will be ignored, please remove it to avoid potential confusion
WARN[0000] /home/yoh/proj/dandi/dandi-archive/docker-compose.override.yml: the attribute `version` is obsolete, it will be ignored, please remove it to avoid potential confusion
[+] Creating 3/0
✔ Container dandi-archive-minio-1 Running 0.0s
✔ Container dandi-archive-rabbitmq-1 Running 0.0s
✔ Container dandi-archive-postgres-1 Running 0.0s
❯ ls -ld 123
-rw-r--r-- 1 yoh root 0 Oct 28 12:18 123 so just add |
note: might need to clean up "root owned" artifacts from prior runs of tox first, and then it seems to work just fine
|
This mostly works--depending on how the container image was built you can still run into problems. Plus, having to add the
This is really the core of the issue--having to remember the But I'm experimenting with moving these things to the build phase so that the regular commands "just work". Stand by, I have an incoming fix for all this (I hope). |
The core issue is :
and the fact that "Docker containers" are not animate, they do not run themselves. They are executed by people who run them and as a result running dev tools as root. It is just a fact.
It is something to remember about and to not execute In the longer run it might be better to look to switch to podman and having fake root inside the container space so touched bind-mounted files would still be fine. I have expressed my opinion ("never run dev tools as root" - whether through |
I did not mean to be dismissive--I was just pointing out a reality of how most people run Docker containers. As I said, I'm working on a solution that addresses this issue head on (it is similar to what you suggested, but even better). |
This extends #2045 to use the same pre-commit hooks to perform linting and formatting via tox.ini. In particular, this "refactors" the specification of what versions of codespell and ruff to use to just one place: the pre-commit config file.
The variants of the pre-commit hooks that are needed to replace the tox.ini commands are listed in the pre-commit config using the
manual
stage specifier with appropriate aliases, meaning they will not be invoked as a git hook ever, and enabling them to be invoked by name in the tox config.There are some quirks w.r.t. running this via Docker:
.git
directory are not those ofroot
, using git or pre-commit (which relies on git) from the container causes pre-commit to sayAn error has occurred: FatalError: git failed. Is it installed, and are you in a Git repository directory?
(Running a command likegit log
inside the container produces the related error messagedetected dubious ownership in repository
.). This can be mitigated by runninggit config --global --add safe.directory /opt/django-project
from inside the container..tox
directory and trying again.Essentially, it's annoying to switch back and forth between the two modes. But in reality, most devs will probably choose one or the other approach (recommended would be to run these commands outside the container).
Improvements to this setup are welcome.