feat(runtime): NVFP4 module split, CUBIN opt-in, col-copy guard, f16out kernels#71
Merged
Merged
Conversation
TheTom
marked this pull request as ready for review
July 23, 2026 20:49
…rename Add ffai_moe_gather_q4, the no-activation sibling of the ReLU2 gather, so gated (SwiGLU) MoE paths can gather the gate and up expert stacks with in-kernel expert indexing. Add ffai_cast_f32_bf16 / ffai_cast_bf16_f32 elementwise casts for bf16 residual streams. Rename the misleading d_f32 parameter to d_f16 on the five kernels whose per-block scales are f16 (gemv_q4 coalesced/relu2/accum, moe_gather_q4 relu2/down) and document the scale dtype contract on each kernel in the family; uploading f32 scale bytes into these silently reinterprets them as garbage f16 and inflates every output.
…ifiers Mechanical cargo fmt over the crate (the CUDA feature line had never run through repo CI, so format debt across many files surfaces on the first PR from it) plus a typos config accepting the intentional short accumulator/nibble names (ba/bb/na/nb) in the hand-rolled GEMV kernels. No behavior change; cargo check clean.
Repo-wide cargo fmt, clippy fixes across the codegen crates (collapsed ifs, dropped useless format! calls, allow on the 9-arg spirv emit_op), typos config for the intentional optin identifiers, and the missing metaltile-core/metaltile-codegen dev-dependencies that the in-module smoke tests in metaltile-std reference (this also un-breaks the full test-target build, which previously failed with unresolved-crate errors before any test ran). No behavior change.
The CI clippy gate runs with --all-features, which pulls the cuda and vulkan device modules into the lint set: allow attrs on the FFI-handle wrappers (not_unsafe_ptr_arg_deref on graph_launch/_batch, dead_code on the CUDA FFI constant surface, too_many_arguments on the grouped-CUTLASS entry), a dropped never-read init, and doc-comment list formatting. Typos config gains dout/ERRO (d-out signal name, log-level tag). No behavior change.
The typos gate reads .github/configs/typos-cli.toml explicitly; add the short GEMV accumulator/nibble identifiers (ba/bb/na/nb) and the dout/erro/optin word stems there and drop the stray root config the checker never consults.
…ut kernels Re-lands the CUDA runtime surface as the committed split (mod.rs plus the nvfp4_moe module) after the working-tree monolith was lost; the batched-Marlin entry points that existed only in that monolith are stubbed in marlin_pending.rs with clear errors (NemotronH-only path, reconstruction notes inline). NVRTC: PTX with driver JIT stays the default (A/B on GB10 measured it slightly ahead for latency-bound decode kernels); MT_NVRTC_NATIVE=1 opts into native-arch SASS via nvrtcGetCUBIN. strided_col_copy gains a bounds guard and ceiling-division grid so non-64-multiple shapes stop silently dropping tail elements (latent, no live caller affected; regression test included). Also lands the f16-out RMSNorm and gated-group-norm kernel variants that previously existed only on the GB10 working tree.
…ormat names The recovered module split predated the workspace fmt sweep; bring it in line. The typos gate flagged the ue4m3/ue8m0 unsigned microscale format names (bare and embedded in FFI constant names); scope an ignore regex to that token shape rather than allowlisting the bare fragment.
TheTom
force-pushed
the
tom/iron-runtime-recovery
branch
from
July 23, 2026 20:51
f72d51b to
150a4ac
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Runtime-side companion to the Laguna prefill PR in the FFAI tree (thewafflehaus/butter#70).
Validation
Lesson encoded
The monolith loss came from a blind cross-machine sync; the recovery plan and the both-trees-identical end state are documented in the working notes. Commit early on shared branches.