Conversation
- moe_mandatory: laguna is sigmoid-routed MoE with a mandatory shared expert; testing it as dense fails on required expert hparams - fixture: write expert_shared_feed_forward_length, expert_weights_scale and expert_weights_norm for MoE configs (laguna requires all three) - laguna: when tensor metadata is unavailable (metadata-only loads used by test-llama-archs) default the attention gate width to per-head; real files with a missing gate still fail in required create_tensor - model-saver: n_ff_chexp was written under the shared-expert FF key, overwriting n_ff_shexp with 0 on save/reload roundtrip; write it under EXPERT_CHUNK_FEED_FORWARD_LENGTH where it belongs
build-vulkan.yml only fired on PRs touching ggml-vulkan sources and dev-build.yml only on C/C++ changes, but their jobs are required status checks on dev -- any PR outside those paths (like this one) hangs on 'Expected -- waiting for status to be reported' with no way to satisfy it. Drop the paths filters from the pull_request triggers (push triggers keep theirs) and let dev-build also run on PRs into master so the dev -> master promotion PR reports the same checks.
tests: fix test-llama-archs failure on laguna
…in archives ggml already implements quantize_nvfp4 (incl. imatrix) and ggml_quantize_chunk dispatches it, but neither llama-quant.cpp nor the quantize tool mapped LLAMA_FTYPE_MOSTLY_NVFP4 to it, so NVFP4 GGUFs could only be obtained by converting pre-quantized checkpoints. Wire the ftype through and list NVFP4 in llama-quantize. Also package llama-quantize into the linux/macos dev and release archives (windows already ships it via Release\*) so a downloaded dev build can produce test quants without a local checkout. Verified locally: SmolLM2-135M f16 -> NVFP4 (259MB -> 88MB), coherent output via llama-cli and llama-server (dev-latest binary) on CPU.
quantize: expose NVFP4 as a quantization target
Linux until now shipped only the Vulkan backend; NVIDIA users got the dequant path and no access to native CUDA kernels (incl. FP4 on Blackwell). Add a CUDA 13.3 build mirroring the Windows cuda-13.3 variant: GGML_BACKEND_DL keeps it a dlopen'd libggml-cuda.so next to the CPU variants, cudart/cublas are bundled like the Windows DLLs, archs cover A100/RTX30/RTX40/H100/RTX50 (80;86;89;90;120).
…p dotted display name
ci: add linux-x64-cuda-13.3 build (dev + release)
On-demand (workflow_dispatch) + nightly: rents the cheapest matching GPU, downloads the released archive, quantizes a tiny f16 model to NVFP4 with the shipped llama-quantize, serves it with -ngl 99, asserts a coherent answer, runs llama-bench and asserts the GPU backend actually did the work -- a silent CPU fallback fails the run. Result is posted as a non-required commit status gpu-smoke/<backend> on the released commit. Deliberately NOT a required PR check: spot GPU rental is slow, nondeterministic and costs money. Hardening baked in from manual runs: X11 client libs + LunarG loader for the NVIDIA Vulkan ICD in headless containers, nohup+poll against vast hosts dropping long ssh sessions, no grep|head under pipefail, box is destroyed in an always() step.
ci: GPU smoke test (vast.ai) for released backends
The CUDA device line in the bench log is 'Device 0: NVIDIA ...' with no 'cuda' on the same line, so the combined regex never matched and a fully working CUDA run (60k t/s pp512 on an RTX 5090) was reported as FAIL. Also the tg128 extraction read the test-name column instead of the t/s column. Both asserts validated against the real bench.log from that run.
smoke: fix CUDA device assert and tg128 column
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Promotion of the stabilized dev branch to master. Contents since
b75801d55:Fixes
llama_model_saverwroten_ff_chexpunder the shared-expert key, zeroingn_ff_shexpon save/reload roundtrip — fixed. Un-redsubuntu-llvmpipe/CI (cpu)on master.pull_requesttriggers) — no more PRs stuck in "Expected".Features
llama-quantize model-f16.gguf out.gguf NVFP4;llama-quantizenow ships in linux/macos archives (windows already had it).libggml-cuda.sonext to the CPU variants, cudart/cublas bundled, archs 80/86/89/90/120 (A100 / RTX 30 / RTX 40 / H100+H200 / RTX 50).backend-smoke.yml— on-demand + nightly smoke of released archives on rented vast.ai GPUs; asserts coherent NVFP4 generation AND that the GPU backend actually did the work. Posts non-requiredgpu-smoke/<backend>commit statuses. The Run workflow button becomes available once this PR lands on master.Verified on a rented RTX 5090 (dev-latest, f2757df)
Both legs: NVFP4 quantize with the shipped tool, coherent generation, GPU-actually-used assert green (after #42).
Release
b10018-1.0.1will be cut from master after this merge.