Skip to content

Fix turbo-KV build flag (FA_ALL_QUANTS) + add CUDA→GHCR build workflow#48

Closed
noonghunna wants to merge 1 commit into
Anbeeld:v0.3.0from
noonghunna:ci/cuda-docker-ghcr
Closed

Fix turbo-KV build flag (FA_ALL_QUANTS) + add CUDA→GHCR build workflow#48
noonghunna wants to merge 1 commit into
Anbeeld:v0.3.0from
noonghunna:ci/cuda-docker-ghcr

Conversation

@noonghunna

Copy link
Copy Markdown

Two small additions to make v0.3.0 pullable as a CUDA Docker image, in support of #39 / the club-3090 discussion where you asked dual+ GPU folks to test v0.3.0. The biggest barrier to community multi-GPU reports right now is that everyone has to compile from source — a published image turns that into docker pull.

1. .devops/cuda.Dockerfile — enable FA_ALL_QUANTS (one line)

The CUDA build currently omits -DGGML_CUDA_FA=ON -DGGML_CUDA_FA_ALL_QUANTS=ON. These are required for the turbo / TCQ KV cache types (turbo2/3/4, turbo2_tcq, turbo3_tcq) — without them the image builds fine but those cache types fail at runtime. Your README/CLAUDE.md already document these flags as required; this just brings the Dockerfile in line so a docker build matches a source build. Worth landing on its own even if you rework the CI part.

2. .github/workflows/build-cuda-docker.yml — CUDA image → your GHCR

  • Builds the server target and pushes to your repo's GHCR (ghcr.io/<owner>/beellama.cpp) via the built-in GITHUB_TOKEN — no secrets to set up.
  • Triggers: nightly cron, push to v0.3.0 (path-filtered to build-affecting files), and manual dispatch (arch selectable).
  • Tags: a rolling :server-cuda-v0.3.0 (always latest) plus an immutable :server-cuda-v0.3.0-<sha> (per-commit, for reproducible bug reports).
  • Nightly skip-guard: if v0.3.0 hasn't moved since the last build, it skips the redundant recompile (only push / manual dispatch force a rebuild).
  • Frees disk on the hosted runner (CUDA builds are big) and caches layers between runs.

Three operational notes:

  • Keep the workflow on your default branch — GitHub only fires scheduled workflows from the default branch (the checkout still builds v0.3.0).
  • Flip the GHCR package to public after the first push so testers can pull without auth.
  • Arch scope — defaulted to the full 86;89;120 so one image covers 3090/4090/5090 (the cards you're recruiting). That's heavy for a free hosted runner (disk + time); if it doesn't fit, the workflow_dispatch input lets you drop an arch, or swap runs-on: to a larger / self-hosted runner (commented inline).

Totally your call on whether/how to take this — happy to adjust anything.

In the meantime we're hosting an unofficial multi-arch (sm_86/89/120) snapshot of v0.3.0 just so the testers in the discussion have something to pull this week. It's a point-in-time snapshot pinned to one commit, though — it won't track your ongoing v0.3.0 work. This workflow is the clean way to make a nightly that does auto-track v0.3.0, which makes the most sense under your ownership of the branch + registry.

The CUDA Dockerfile omits -DGGML_CUDA_FA=ON -DGGML_CUDA_FA_ALL_QUANTS=ON,
required at runtime for the turbo/TCQ KV cache types. Also add a nightly +
on-push GitHub Actions workflow that builds the server target and pushes a
multi-arch (sm_86/89/120) CUDA image to GHCR, so testers can pull instead of
compiling from source. Context: issue Anbeeld#39 / club-3090 discussion.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@github-actions github-actions Bot added the devops label Jun 1, 2026
@Anbeeld

Anbeeld commented Jun 1, 2026

Copy link
Copy Markdown
Owner

Implemented CI/CD for both build and Docker images, but in a different way, on-push for main and versioned branches. Fixed missing quant args and a number of other issues with builds across various platforms. Currently v0.3.0 seems to have a successful build, please try if Docker works with no issues.

Closing this PR as the end goal should be achieved, if there are still some issues I'll do follow-ups.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants