feat(gemv_q8): plain and SwiGLU-fused MoE gather kernels, bf16 casts, f16-scale rename#69
Merged
Merged
Conversation
…rename Add ffai_moe_gather_q4, the no-activation sibling of the ReLU2 gather, so gated (SwiGLU) MoE paths can gather the gate and up expert stacks with in-kernel expert indexing. Add ffai_cast_f32_bf16 / ffai_cast_bf16_f32 elementwise casts for bf16 residual streams. Rename the misleading d_f32 parameter to d_f16 on the five kernels whose per-block scales are f16 (gemv_q4 coalesced/relu2/accum, moe_gather_q4 relu2/down) and document the scale dtype contract on each kernel in the family; uploading f32 scale bytes into these silently reinterprets them as garbage f16 and inflates every output.
…ifiers Mechanical cargo fmt over the crate (the CUDA feature line had never run through repo CI, so format debt across many files surfaces on the first PR from it) plus a typos config accepting the intentional short accumulator/nibble names (ba/bb/na/nb) in the hand-rolled GEMV kernels. No behavior change; cargo check clean.
Repo-wide cargo fmt, clippy fixes across the codegen crates (collapsed ifs, dropped useless format! calls, allow on the 9-arg spirv emit_op), typos config for the intentional optin identifiers, and the missing metaltile-core/metaltile-codegen dev-dependencies that the in-module smoke tests in metaltile-std reference (this also un-breaks the full test-target build, which previously failed with unresolved-crate errors before any test ran). No behavior change.
The CI clippy gate runs with --all-features, which pulls the cuda and vulkan device modules into the lint set: allow attrs on the FFI-handle wrappers (not_unsafe_ptr_arg_deref on graph_launch/_batch, dead_code on the CUDA FFI constant surface, too_many_arguments on the grouped-CUTLASS entry), a dropped never-read init, and doc-comment list formatting. Typos config gains dout/ERRO (d-out signal name, log-level tag). No behavior change.
The typos gate reads .github/configs/typos-cli.toml explicitly; add the short GEMV accumulator/nibble identifiers (ba/bb/na/nb) and the dout/erro/optin word stems there and drop the stray root config the checker never consults.
TheTom
marked this pull request as ready for review
July 23, 2026 12:10
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Kernel-side companion to the Laguna decode PR in the FFAI tree (thewafflehaus/FFAI#69).
Validation
Perf context
These kernels carry the Laguna MoE decode path that reaches 36.5 tok/s single-stream on GB10 (reference implementation on identical weights and box: 21.24).