feat(docker): ROCm AMD image, device_utils, CI, and Ruff gate#98
feat(docker): ROCm AMD image, device_utils, CI, and Ruff gate#98BlackFoxgamingstudio wants to merge 1 commit into
Conversation
…ning Add optional AMD GPU support via docker/Dockerfile.rocm (rocm-runtime and rocm-vnc targets), Compose profile rocm, and device_utils.setup_rocm_env() hooked from main. Include mocked ROCm tests, rocm-ci workflow, and docs. Add merge-gate Ruff workflow plus [tool.ruff] in pyproject.toml; apply ruff format and fix lint across the tree. Fix .gitignore so tests/test_*.py under tests/ are tracked (/test_*.py for root scripts only). Pin filebrowser get.sh to a fixed commit in both Dockerfiles; tighten supervisord/entrypoint env for CORRIDORKEY_CONTAINER_MODE; document security and Compose profiles in docker/README and ROCm docs. Made-with: Cursor
|
Nice work - I'll try and make some time to test it. It might be worth separating out Ruff (and anything else not directly related to ROCm support if it's there) to another PR - that way those bits can be applied without someone requiring AMD graphics. With PRs it tends to be best to do one per feature / topic. |
|
Hello, I am in the process of testing this. Is there something in this perticular version that should have been used? if so, this version exactly does not exist in the docker repo anymore. |
AMD ROCm Docker — Tested on RX 6700 XT (RDNA2) — Fixes RequiredI successfully tested PR #98 (ROCm Docker support) on a RX 6700 XT (RDNA2 / gfx1031) running Debian 13 with Docker. No ROCm or HIP installed on the host — everything is containerized. Several fixes were required to get full GPU-accelerated inference and proper FFmpeg hardware decoding working. Documenting everything here for other AMD users and to help improve the PR. System Info Dockerfile.rocm FixesBase image tag doesn't exist on Docker Hub, and # Line 9 — fix base image
FROM rocm/pytorch:latest AS rocm-base
# Line 20 — fix package name
libgl1 \PyTorch .venv Ignores System ROCm TorchThe FFmpeg Too Old — No VAAPIThe base image ships with FFmpeg 6.1.1 but the app requires 7.0+. The apt default for Ubuntu 24.04 doesn't provide a newer version. Installing from Docker Compose Issues
Working SetupAll runtime fixes applied via an init script mounted into the container:
#!/bin/bash
set -e
# Fix PyTorch: use ROCm system torch instead of CPU .venv torch
sed -i 's/include-system-site-packages = false/include-system-site-packages = true/' \
/opt/corridorkey/.venv/pyvenv.cfg 2>/dev/null || true
echo "/opt/venv/lib/python3.12/site-packages" > \
/opt/corridorkey/.venv/lib/python3.12/site-packages/rocm-torch.pth
uv pip uninstall torch torchvision torchaudio \
--python /opt/corridorkey/.venv/bin/python 2>/dev/null || true
# Fix FFmpeg: install 7.x with VAAPI from PPA
if ! ffmpeg -version 2>&1 | grep -q "ffmpeg version 7"; then
echo "Installing FFmpeg 7.x with VAAPI..."
apt-get update -qq
apt-get install -y --no-install-recommends software-properties-common
add-apt-repository -y ppa:ubuntuhandbook1/ffmpeg7
apt-get update -qq
apt-get install -y ffmpeg mesa-va-drivers libva2 libva-drm2
rm -rf /var/lib/apt/lists/*
fi
rm -f /usr/local/bin/ffmpeg /usr/local/bin/ffprobe
# Fix FFmpeg threading
sed -i 's/"-vsync",/"-threads", "0", "-vsync",/g' \
/opt/corridorkey/backend/ffmpeg_tools/extraction.py 2>/dev/null || true
exec /usr/bin/supervisord -c /etc/supervisor/conf.d/corridorkey-vnc.conf
services:
ez-corridorkey:
image: ez-corridorkey-rocm-vnc
container_name: ez-corridorkey-rocm
ipc: host
entrypoint: ["/bin/bash", "/init-rocm.sh"]
devices:
- /dev/kfd
- /dev/dri
group_add:
- video
- 992 # render GID — check yours with: getent group render
environment:
- DISPLAY=:1
- QT_QPA_PLATFORM=xcb
- CORRIDORKEY_CONTAINER_MODE=1
- CORRIDORKEY_RESOLUTION=1920x1080x24
- HSA_OVERRIDE_GFX_VERSION=10.3.0
- HIP_VISIBLE_DEVICES=0
- HF_HUB_DISABLE_PROGRESS_BARS=1
ports:
- "6080:6080"
- "6081:6081"
- "5900:5900"
volumes:
- ./init-rocm.sh:/init-rocm.sh:ro
- corridorkey_models_corridorkey:/opt/corridorkey/CorridorKeyModule/checkpoints
- corridorkey_models_gvm:/opt/corridorkey/gvm_core/weights
- corridorkey_models_videomama:/opt/corridorkey/VideoMaMaInferenceModule/checkpoints
- corridorkey_hf_cache:/root/.cache/huggingface/hub
- ./ClipsForInference:/opt/corridorkey/ClipsForInference
- ./Projects:/opt/corridorkey/Projects
- ./Output:/opt/corridorkey/Output
- ./corridorkey_config:/root/.config/corridorkey
- ./corridorkey_logs:/opt/corridorkey/logs
security_opt:
- seccomp=unconfined
stdin_open: true
tty: true
restart: unless-stopped
volumes:
corridorkey_models_corridorkey:
name: corridorkey_models_corridorkey
corridorkey_models_gvm:
name: corridorkey_models_gvm
corridorkey_models_videomama:
name: corridorkey_models_videomama
corridorkey_hf_cache:
name: corridorkey_hf_cacheBuild & run: git clone https://github.com/edenaion/EZ-CorridorKey.git
cd EZ-CorridorKey
git fetch origin pull/98/head:test-rocm
git checkout test-rocm
sed -i 's|rocm/pytorch:rocm6.1-py3.10-ubuntu22.04|rocm/pytorch:latest|; s|libgl1-mesa-glx|libgl1|' docker/Dockerfile.rocm
docker build -f docker/Dockerfile.rocm --target rocm-vnc -t ez-corridorkey-rocm-vnc .
cd docker-rocm # directory containing compose + init script
docker compose -f docker-compose-rocm.yml up -d
|
|
Quick update — still need to test the rest of the pipeline past frame extraction, but I'm calling it a night. The EXR frame extraction is taking forever (single-threaded bottleneck in FFmpeg's image sequence writer). I have a lead on improving that with additional threading flags, will test tomorrow morning. Other than that, nothing suggests there will be any issues downstream. The entire application behaves as if it has access to an NVIDIA GPU — it just happens to be going through ROCm instead. Pretty seamless once all the fixes above are in place. |
|
@le-patenteux Awesome, well done! Please keep me posted as you're the link to the AMD community currently. Appreciate your efforts <3 |
Related to #92.
Summary
Adds optional AMD GPU (ROCm) support via
docker/Dockerfile.rocm:rocm-runtime(default build target):uv run python main.py.rocm-vnc: same baked environment plus noVNC / x11vnc / filebrowser / supervisord (parity with the defaultdocker/DockerfileUX).ROCm PyTorch +
pytorch-triton-rocmare applied only inside the image (uv pip install … --reinstallafteruv sync). Norocmoptional extra inpyproject.toml.Also includes
device_utils.setup_rocm_env()(HIP detection,HSA_OVERRIDE_GFX_VERSION, stderr logging beforesetup_logging),main._try_setup_rocm_env(), mocked ROCm tests, Compose profilerocm,.github/workflows/rocm-ci.yml,device_utilsin wheel/sdist via hatchforce-include,torch-backend = "auto"under[tool.uv.pip], a Ruff workflow (ruff.yml) +[tool.ruff], pinned filebrowserget.shin both Dockerfiles,.gitignorefix sotests/test_*.pyundertests/are tracked, and docs underdocs/ROCm_Setup.md/docs/ROCm_PR_handoff.md.How to test
Docker (optional; large pull):
Docs
docs/ROCm_Setup.mddocs/ROCm_PR_handoff.mdNote: The contributor does not have AMD hardware; ROCm paths are designed for review and validation by someone with a suitable GPU.