Unspecified CUDA launch failure on L4 GPU (sm89) with pv_accum_dtype="fp32+fp16"

[Li1177]

Describe the bug
After updating to the latest version of SageAttention, running the attention module on NVIDIA L4 (sm89) GPU in Google Colab consistently crashes with the following CUDA error:
terminate called after throwing an instance of 'c10::Error'
what(): CUDA error: unspecified launch failure
Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.

To Reproduce
Steps to reproduce the behavior:

Use Google Colab Pro+ with default L4 GPU (sm89), CUDA 12.4

Install latest SageAttention via:

git clone https://github.com/thu-ml/SageAttention
pip install -e ./SageAttention

Integrate with a Stable Diffusion pipeline (e.g. ComfyUI with SDXL)

Trigger an attention module pass using the L4 GPU

Observe the runtime crash

Expected behavior
Expected it to run as before (with pv_accum_dtype="fp32+fp32" fallback), or at least raise a user-facing error or warning.

Actual behavior
The attention module attempts to run with:

pv_accum_dtype="fp32+fp16"

on a GPU (L4 / CUDA 12.4) that cannot handle this precision path, leading to an unspecified launch failure.

Relevant code
In sageattention/core.py, the logic was changed:

• pv_accum_dtype="fp32+fp32"

• pv_accum_dtype="fp32+fp16"

And the CUDA version fallback was commented out:

• if(cuda_major_version, cuda_minor_version) < (12, 8) and pv_accum_dtype == 'fp32+fp16':

• pv_accum_dtype = 'fp32+fp32'
  

Environment (please complete the following information):

GPU: NVIDIA L4 (sm89)

CUDA version: 12.4

PyTorch: 2.6.0

TorchVision: 0.21.0

Python: 3.11 (default on Colab)

Platform: Google Colab Pro+

Workaround
Manually reverting the line back to:

pv_accum_dtype="fp32+fp32"

resolves the crash on L4 GPUs.

Proposed fix

Restore the fallback logic based on CUDA version

Or provide a safe config flag to manually set pv_accum_dtype

Or at least log a runtime warning when CUDA < 12.8

Additional context
This issue affects a wide range of Colab users who use L4 as the default Pro+ GPU. The sudden crash broke workflows that had been stable for months.

Thanks again for your great work on this module! I hope this helps improve usability across different CUDA versions.

[umerkay]

I can confirm that the same problem occurs on cuda 12.5. It should fallback with a warning. Also, the readme does not specifiy that cuda 12.8 is necessary.

[umerkay]

Make a fork and PR
#202 This works for me on CUDA 12.5