TurboQuant b10269-1.5.1

TurboQuant b10269-1.5.1

Fixed

• Ling-3.0-flash (BailingMoeV3) no longer emits garbage token bursts. The
  model is trained with clamped SwiGLU activations in its late layers, and the
  per-layer limits live in config.json under expert_swiglu_limit_list and
  share_expert_swiglu_limit_list. The public HF modeling code ignores those
  keys and so did this port, which caused deterministic transient logit
  collapse - output like count += 1eville dropped into otherwise fine
  generations. Measured at roughly -20 pass@1 on HumanEval (72.6% -> 93%+ with
  the fix); the garbage-token repro is eliminated.

Notes

• Re-convert your Ling-3.0-flash GGUF to get the fix. The clamp limits are
  written by the converter into two new KVs ({arch}.swiglu_clamp_exp and
  {arch}.swiglu_clamp_shexp); a GGUF produced before this release does not
  carry them, and the runtime then defaults to no clamping. Re-download the
  quant or re-run conversion/bailingmoe.py.
• Both KVs are optional and default to zero, so existing GGUFs and every other
  architecture are unaffected. The graph needed no change - the SwiGLU clamp
  branches in build_ffn / build_moe_ffn already trigger on a nonzero
  per-layer limit, matching the vLLM SwigluStepAndMul semantics.

Downloads

Backend	Asset
Linux x64 CPU only	llama-turboquant-linux-x64-cpu.tar.gz
Linux x64 Vulkan (+ portable CPU)	llama-turboquant-linux-x64-vulkan.tar.gz
Linux x64 CUDA 12.4, older drivers (+ portable CPU)	llama-turboquant-linux-x64-cuda-12.4.tar.gz
Linux x64 CUDA 13.3 (+ portable CPU)	llama-turboquant-linux-x64-cuda-13.3.tar.gz
Linux arm64 CUDA 13.3, DGX Spark / GB10 (+ portable CPU)	llama-turboquant-linux-arm64-cuda-13.3.tar.gz
Linux x64 AMD ROCm, RDNA2-RDNA4 (+ portable CPU)	llama-turboquant-linux-x64-rocm.tar.gz
Windows x64 CPU	llama-turboquant-windows-x64-cpu.zip
Windows x64 Vulkan	llama-turboquant-windows-x64-vulkan.zip
Windows x64 CUDA 12.4	llama-turboquant-windows-x64-cuda-12.4.zip
Windows x64 CUDA 13.3	llama-turboquant-windows-x64-cuda-13.3.zip
macOS ARM64 (Metal, signed + notarized)	llama-turboquant-macos-arm64.zip

The AMD ROCm archive targets RDNA2 through RDNA4 (gfx1030/1100/1101/1102/1151/1200/1201) and CDNA (gfx90a, gfx942 -- Instinct MI200/MI300) in a single archive, and needs the ROCm runtime installed on the system. Older GCN cards: use the Vulkan build.

The Linux arm64 CUDA archive is built for NVIDIA DGX Spark (GB10, sm_121); other arm64 NVIDIA machines (GH200, GB200, Thor) run it too but JIT the kernels from PTX on first launch.

All commits since b10269-1.5.0

• 📝 docs(changelog): add the b10269-1.5.1 section
• release: b10269-1.5.1
• bailingmoe3: apply trained SwiGLU clamps (expert_swiglu_limit_list)

What every archive contains, and how versions work

• llama-server with --cache-type-k turbo3 / turbo4 support
• llama-cli, llama-bench, llama-perplexity, llama-quantize

Built from master at commit cd5609390, based on upstream llama.cpp b10269.

<upstream-base>-<fork-semver>: b10269 is the upstream llama.cpp build this fork is based on, 1.5.1 is the TurboQuant fork version. llama-server --version reports version: b10269-1.5.1.

Dev latest (4e7f9e950, 2026-08-06)

Rolling dev build from dev at commit 4e7f9e950 (2026-08-06).

Staging channel — not for production. Every push to dev overwrites this release.

macOS binaries are signed but not notarized; after unpacking run xattr -dr com.apple.quarantine build/. Stable releases are fully notarized.

TurboQuant b10269-1.5.0

TurboQuant b10269-1.5.0

Added

• NVIDIA DGX Spark (GB10) support. New archive
  llama-turboquant-linux-arm64-cuda-13.3, built natively for aarch64 with
  CUDA 13.3 and sm_121 SASS. Other arm64 NVIDIA machines (GH200, GB200, Jetson
  Thor) run it too, JITing the kernels from PTX on first launch. This is the
  first Linux arm64 build the fork ships — until now arm64 meant macOS only.
• BailingMoeV3 (Ling 3.0) architecture support, including the KDA gate
  handling.

Changed

• Linux CUDA archives are roughly half the size - 1657 -> 956 MB (12.4) and
  1879 -> 1028 MB (13.3) measured across both the .zip and .tar.gz. The zips
  were storing libcublas.so -> .so.13 -> .so.13.5.1.27 as three full copies
  because zip followed the symlinks.
• CUDA 13.3 builds ship Ampere PTX (80-virtual). A100/H100/B200 were
  falling back to the Turing PTX floor, which silently disabled cp.async and
  the Ampere MMA path - both gated on __CUDA_ARCH__ >= 800. Those cards get
  Ampere-class kernels now. No architecture lost support in this release.
• Windows CUDA builds got their architecture lists pinned, all runner cores, a
  ccache that can actually hold a CUDA build, and 7-Zip instead of
  Compress-Archive. Release turnaround drops accordingly.

Notes

• The DGX Spark archive has not yet been validated on real GB10 hardware -
  it is built and arch-checked in CI (cuobjdump asserts sm_121 SASS is
  present), but nobody has run it on a Spark yet. Treat this one as beta and
  report back.
• The CUDA 13.3 archives now use -compress-mode=size. Kernel SASS is
  unchanged and inference speed is unaffected; the fatbin is decompressed once
  at module load. It needs a driver from the CUDA 12.4 era or newer.

Downloads

Backend	Asset
Linux x64 CPU only	llama-turboquant-linux-x64-cpu.tar.gz
Linux x64 Vulkan (+ portable CPU)	llama-turboquant-linux-x64-vulkan.tar.gz
Linux x64 CUDA 12.4, older drivers (+ portable CPU)	llama-turboquant-linux-x64-cuda-12.4.tar.gz
Linux x64 CUDA 13.3 (+ portable CPU)	llama-turboquant-linux-x64-cuda-13.3.tar.gz
Linux arm64 CUDA 13.3, DGX Spark / GB10 (+ portable CPU)	llama-turboquant-linux-arm64-cuda-13.3.tar.gz
Linux x64 AMD ROCm (+ portable CPU)	llama-turboquant-linux-x64-rocm.tar.gz
Windows x64 CPU	llama-turboquant-windows-x64-cpu.zip
Windows x64 Vulkan	llama-turboquant-windows-x64-vulkan.zip
Windows x64 CUDA 12.4	llama-turboquant-windows-x64-cuda-12.4.zip
Windows x64 CUDA 13.3	llama-turboquant-windows-x64-cuda-13.3.zip
macOS ARM64 (Metal, signed + notarized)	llama-turboquant-macos-arm64.zip

The AMD ROCm archive targets RDNA2 through RDNA4 (gfx1030/1100/1101/1102/1151/1200/1201) and CDNA (gfx90a, gfx942 -- Instinct MI200/MI300) in a single archive, and needs the ROCm runtime installed on the system. Older GCN cards: use the Vulkan build.

The Linux arm64 CUDA archive is built for NVIDIA DGX Spark (GB10, sm_121); other arm64 NVIDIA machines (GH200, GB200, Thor) run it too but JIT the kernels from PTX on first launch.

All commits since b10269-1.4.0

• release: b10269-1.5.0
• 📝 docs(release): generate release notes from CHANGELOG.md
• 🐛 fix(ci): keep the windows job names off the required-check list
• ⚡ perf(ci): cut CUDA build time and archive size
• 🔨 feat(ci): linux-arm64 CUDA 13.3 build for DGX Spark
• 🔧 fix(tests): skip BailingMoeV3 in the synthetic arch fixture
• 🔧 fix(model): reuse the shared KDA gate lower bound key
• 🔧 fix(model): use the safe-gate form for the BailingMoeV3 KDA decay
• 🔨 feat(model): add BailingMoeV3 (Ling 3.0) support

What every archive contains, and how versions work

• llama-server with --cache-type-k turbo3 / turbo4 support
• llama-cli, llama-bench, llama-perplexity, llama-quantize

Built from master at commit 83801dfe8, based on upstream llama.cpp b10269.

<upstream-base>-<fork-semver>: b10269 is the upstream llama.cpp build this fork is based on, 1.5.0 is the TurboQuant fork version. llama-server --version reports version: b10269-1.5.0.

TurboQuant b10269-1.4.0

TurboQuant KV Cache b10269-1.4.0

Built from master at commit 074bf826e, based on upstream llama.cpp b10269.

Feaures

• DeepSeek V4 Flash 0731 support & speedup (+custom ops CUDA kernels)
• Kimi K3 + vision full support
• RDNA2-4 + CDNA support for AMD GPUs in linux x64 release
• commits sync from upstream

What's included

• llama-server with --cache-type-k turbo3 / turbo4 support
• llama-cli, llama-bench, llama-perplexity, llama-quantize
• All supported backends in one release:

Backend	Asset
Linux x64 CPU only	llama-turboquant-linux-x64-cpu.tar.gz
Linux x64 Vulkan (+ portable CPU)	llama-turboquant-linux-x64-vulkan.tar.gz
Linux x64 CUDA 12.4, older drivers (+ portable CPU)	llama-turboquant-linux-x64-cuda-12.4.tar.gz
Linux x64 CUDA 13.3 (+ portable CPU)	llama-turboquant-linux-x64-cuda-13.3.tar.gz
Linux x64 AMD ROCm (+ portable CPU)	llama-turboquant-linux-x64-rocm.tar.gz
Windows x64 CPU	llama-turboquant-windows-x64-cpu.zip
Windows x64 Vulkan	llama-turboquant-windows-x64-vulkan.zip
Windows x64 CUDA 12.4	llama-turboquant-windows-x64-cuda-12.4.zip
Windows x64 CUDA 13.3	llama-turboquant-windows-x64-cuda-13.3.zip
macOS ARM64 (Metal, signed + notarized)	llama-turboquant-macos-arm64.zip

The AMD ROCm archive targets RDNA2 through RDNA4 (gfx1030/1100/1101/1102/1151/1200/1201) and CDNA (gfx90a, gfx942 -- Instinct MI200/MI300) in a single archive, and needs the ROCm runtime installed on the system. Older GCN cards: use the Vulkan build.

Versioning

<upstream-base>-<fork-semver>: b10269 is the upstream llama.cpp build this fork is based on, 1.4.0 is the TurboQuant fork version. llama-server --version reports version: b10269-1.4.0.

TurboQuant b10018-1.3.0

TurboQuant KV Cache b10018-1.3.0

Built from master at commit 5bc5c248d, based on upstream llama.cpp b10018.

What's included

• llama-server with --cache-type-k turbo3 / turbo4 support
• llama-cli, llama-bench, llama-perplexity, llama-quantize
• All supported backends in one release:

Backend	Asset
Linux x64 CPU only	llama-turboquant-linux-x64-cpu.tar.gz
Linux x64 Vulkan (+ portable CPU)	llama-turboquant-linux-x64-vulkan.tar.gz
Linux x64 CUDA 12.4, older drivers (+ portable CPU)	llama-turboquant-linux-x64-cuda-12.4.tar.gz
Linux x64 CUDA 13.3 (+ portable CPU)	llama-turboquant-linux-x64-cuda-13.3.tar.gz
Linux x64 AMD ROCm, RDNA2-RDNA4 (+ portable CPU)	llama-turboquant-linux-x64-rocm.tar.gz
Windows x64 CPU	llama-turboquant-windows-x64-cpu.zip
Windows x64 Vulkan	llama-turboquant-windows-x64-vulkan.zip
Windows x64 CUDA 12.4	llama-turboquant-windows-x64-cuda-12.4.zip
Windows x64 CUDA 13.3	llama-turboquant-windows-x64-cuda-13.3.zip
macOS ARM64 (Metal, signed + notarized)	llama-turboquant-macos-arm64.zip

The AMD ROCm archive targets RDNA2 through RDNA4 (gfx1030/1100/1101/1102/1151/1200/1201) and needs the ROCm runtime installed on the system. Older GCN cards: use the Vulkan build.

Versioning

<upstream-base>-<fork-semver>: b10018 is the upstream llama.cpp build this fork is based on, 1.3.0 is the TurboQuant fork version. llama-server --version reports version: b10018-1.3.0.

TurboQuant b10018-1.2.0

TurboQuant KV Cache b10018-1.2.0

Built from master at commit 2144aac6f, based on upstream llama.cpp b10018.

What's included

• llama-server with --cache-type-k turbo3 / turbo4 support
• llama-cli, llama-bench, llama-perplexity, llama-quantize
• All supported backends in one release:

Backend	Asset
Linux x64 Vulkan (+ portable CPU)	llama-turboquant-linux-x64-vulkan.tar.gz
Linux x64 CUDA 13.3 (+ portable CPU)	llama-turboquant-linux-x64-cuda-13.3.tar.gz
Linux x64 AMD ROCm, RDNA2-RDNA4 (+ portable CPU)	llama-turboquant-linux-x64-rocm.tar.gz
Windows x64 CPU	llama-turboquant-windows-x64-cpu.zip
Windows x64 Vulkan	llama-turboquant-windows-x64-vulkan.zip
Windows x64 CUDA 12.4	llama-turboquant-windows-x64-cuda-12.4.zip
Windows x64 CUDA 13.3	llama-turboquant-windows-x64-cuda-13.3.zip
macOS ARM64 (Metal, signed + notarized)	llama-turboquant-macos-arm64.zip

The AMD ROCm archive targets RDNA2 through RDNA4 (gfx1030/1100/1101/1102/1151/1200/1201) and needs the ROCm runtime installed on the system. Older GCN cards: use the Vulkan build.

Versioning

<upstream-base>-<fork-semver>: b10018 is the upstream llama.cpp build this fork is based on, 1.2.0 is the TurboQuant fork version. llama-server --version reports version: b10018-1.2.0.

TurboQuant b10018-1.1.2

TurboQuant KV Cache — b10018-1.1.2

Built from master at commit dc26e6ea4, based on upstream llama.cpp b10018.

What's included

• llama-server with --cache-type-k turbo3 / turbo4 support
• llama-cli, llama-bench, llama-perplexity, llama-quantize
• All supported backends in one release:

Backend	Asset
Linux x64 (Vulkan + portable CPU)	llama-turboquant-linux-x64-vulkan.tar.gz
Linux x64 CUDA 13.3 (+ portable CPU)	llama-turboquant-linux-x64-cuda-13.3.tar.gz
Windows x64 CPU	llama-turboquant-windows-x64-cpu.zip
Windows x64 Vulkan	llama-turboquant-windows-x64-vulkan.zip
Windows x64 CUDA 12.4	llama-turboquant-windows-x64-cuda-12.4.zip
Windows x64 CUDA 13.3	llama-turboquant-windows-x64-cuda-13.3.zip
macOS ARM64 (Metal, signed + notarized)	llama-turboquant-macos-arm64.zip

Versioning

<upstream-base>-<fork-semver>: b10018 is the upstream llama.cpp build this fork is based on, 1.1.2 is the TurboQuant fork version. llama-server --version reports version: b10018-1.1.2.

TurboQuant b10018-1.1.1

TurboQuant KV Cache — b10018-1.1.1

Built from master at commit d4db42d39, based on upstream llama.cpp b10018.

What's included

• llama-server with --cache-type-k turbo3 / turbo4 support
• llama-cli, llama-bench, llama-perplexity, llama-quantize
• All supported backends in one release:

Backend	Asset
Linux x64 (Vulkan + portable CPU)	llama-turboquant-linux-x64-vulkan.tar.gz
Linux x64 CUDA 13.3 (+ portable CPU)	llama-turboquant-linux-x64-cuda-13.3.tar.gz
Windows x64 CPU	llama-turboquant-windows-x64-cpu.zip
Windows x64 Vulkan	llama-turboquant-windows-x64-vulkan.zip
Windows x64 CUDA 12.4	llama-turboquant-windows-x64-cuda-12.4.zip
Windows x64 CUDA 13.3	llama-turboquant-windows-x64-cuda-13.3.zip
macOS ARM64 (Metal, signed + notarized)	llama-turboquant-macos-arm64.zip

Versioning

<upstream-base>-<fork-semver>: b10018 is the upstream llama.cpp build this fork is based on, 1.1.1 is the TurboQuant fork version. llama-server --version reports version: b10018-1.1.1.

TurboQuant b10018-1.1.0

TurboQuant KV Cache — b10018-1.1.0

Built from master at commit 67861a917, based on upstream llama.cpp b10018.

What's new in 1.1.0

• NVFP4 quantization target: llama-quantize model-f16.gguf out.gguf NVFP4 (ggml already had the kernels; the ftype is now wired through). llama-quantize ships in all archives.
• First Linux CUDA backend (linux-x64-cuda-13.3): native CUDA kernels on Linux incl. native FP4 on Blackwell; cudart/cublas bundled; archs 80/86/89/90/120. Verified on an RTX 5090: 60.5k t/s pp512 / 1349 t/s tg128 on an NVFP4 model (2.2-2.4x the Vulkan path).
• GPU smoke CI: released archives are now smoke-tested on rented GPUs (NVFP4 quantize -> serve -> coherence + GPU-actually-used asserts), on demand and nightly.
• Fixes: test-llama-archs laguna failure (MoE-only arch + fixture keys + gate-meta fallback); llama_model_saver wrote n_ff_chexp under the shared-expert key, zeroing n_ff_shexp on save/reload roundtrip.

What's included

• llama-server with --cache-type-k turbo3 / turbo4 support
• llama-cli, llama-bench, llama-perplexity, llama-quantize
• All supported backends in one release:

Backend	Asset
Linux x64 (Vulkan + portable CPU)	llama-turboquant-linux-x64-vulkan.tar.gz
Linux x64 CUDA 13.3 (+ portable CPU)	llama-turboquant-linux-x64-cuda-13.3.tar.gz
Windows x64 CPU	llama-turboquant-windows-x64-cpu.zip
Windows x64 Vulkan	llama-turboquant-windows-x64-vulkan.zip
Windows x64 CUDA 12.4	llama-turboquant-windows-x64-cuda-12.4.zip
Windows x64 CUDA 13.3	llama-turboquant-windows-x64-cuda-13.3.zip
macOS ARM64 (Metal, signed + notarized)	llama-turboquant-macos-arm64.zip

Versioning

<upstream-base>-<fork-semver>: b10018 is the upstream llama.cpp build this fork is based on, 1.1.0 is the TurboQuant fork version. llama-server --version reports version: b10018-1.1.0.

TurboQuant b10018-1.0.0

TurboQuant KV Cache — b10018-1.0.0

Built from master at commit b75801d55, based on upstream llama.cpp b10018.

What's included

• llama-server with --cache-type-k turbo3 / turbo4 support
• llama-cli, llama-bench, llama-perplexity
• All supported backends in one release:

Backend	Asset
Linux x64 (Vulkan + portable CPU)	llama-turboquant-linux-x64-vulkan.tar.gz
Windows x64 CPU	llama-turboquant-windows-x64-cpu.zip
Windows x64 Vulkan	llama-turboquant-windows-x64-vulkan.zip
Windows x64 CUDA 12.4	llama-turboquant-windows-x64-cuda-12.4.zip
Windows x64 CUDA 13.3	llama-turboquant-windows-x64-cuda-13.3.zip
macOS ARM64 (Metal, signed + notarized)	llama-turboquant-macos-arm64.zip

Versioning

<upstream-base>-<fork-semver>: b10018 is the upstream llama.cpp build this fork is based on, 1.0.0 is the TurboQuant fork version. llama-server --version reports version: b10018-1.0.0.