fix(kernels): catch C++ exceptions at the FFI boundary and surface the message#539
Merged
xiaguan merged 4 commits intoJul 4, 2026
Merged
Conversation
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 154764075e
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
1547640 to
b188365
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fixes #520
FlashInfer can throw
std::exceptionfrom host-side dispatch before any CUDA kernel launch. When that exception crosses anextern "C"boundary, the process aborts without surfacing the message. This was hit by Qwen3-14B: 40/8 heads -> GQA group size 5, which is not instantiated byDISPATCH_GQA_GROUP_SIZE.csrc/shared/ffi_guard.cuh+ffi_last_error.cu: guarded FFI entries catch C++ exceptions at the boundary, storewhat()in athread_local, clear stale messages on entry, and return a-1sentinel.-1;ffi_exception_messageis public for direct-FFI callers outside the kernels crate. The qwen35 hd256 prefill path and qwen3 kernel bench now append the message too.SUPPORTED_GQA_GROUP_SIZES(&[1, 2, 3, 4, 8], matching the FlashInfer dispatch table) is exported from the kernels crate; qwen3 config load rejects unsupported group sizes up front.i32at the FFI declaration so the-1sentinel cannot materialize an invalidCUresult.The new FFI test drives decode at group 7 and asserts
-1plus a message naming the group size, instead of a process abort.Test Env
Single GPU (sm_89, x86_64):
unsupported GQA group size 40/8; the attention kernels instantiate [1, 2, 3, 4, 8].kimi-k2andglm52feature builds compile clean with the guard additions (cargo checktargeting sm_90 with the fullmoedependency chain).Type of Change