Skip to content

Issues: flashinfer-ai/flashinfer

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

85 Open 143 Closed
85 Open 143 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Anyway to build flashinfer with pytorch 2.7.0 and cuda 12.8
#922 opened Mar 8, 2025 by ryderhwang
RMSNorm failed with error code no kernel image is available for execution on the device
#920 opened Mar 7, 2025 by nanmi
1
[Bugfix] CUDAGraph Compatibility in AppendPagedKVCacheKernel for Variable-Length Inputs
#919 opened Mar 7, 2025 by SungBalance
1
ValueError: Invalid mode: forward_mode=<ForwardMode.TARGET_VERIFY: 6>
#916 opened Mar 6, 2025 by tchaton
flashinfer.prefill.single_prefill_with_kv_cache meets error when running on A100
#915 opened Mar 6, 2025 by FengzhuoZhang
flashinfer fp8 prefill has no speed up than fp16 in L20
#914 opened Mar 6, 2025 by yongchaoding
flashinfer_kernels.abi3.so: undefined symbol: _ZN5torch3jit17parseSchemaOrNameERKSsb
#911 opened Mar 5, 2025 by Idonthaveaname-wq
int4 kv cache support
#908 opened Mar 4, 2025 by ZL1206
AttributeError: module 'flashinfer' has no attribute 'mla'
#907 opened Mar 3, 2025 by ThomasBaruzier
1
[Feature] Can flashinfer support different custom mask for different attention head?
#906 opened Mar 2, 2025 by yuzhenmao
1
Can't run on CUDA 12.8 with older driver version because of CUTLASS issue
#903 opened Feb 27, 2025 by mickaelseznec
1
Feature Request: DeepGEMM from Deepseek
#899 opened Feb 26, 2025 by haichuan1221
[Tracking Issue] MLA performance tracking
#897 opened Feb 24, 2025 by yzh119
5 of 10 tasks
FlashMLA from DeepSeek
#892 opened Feb 24, 2025 by zhyncs
14
Possible Bug in chain_speculative_sampling: output_emitted_token_num Exceeds Expected Limit
#879 opened Feb 19, 2025 by JaeminK
The current _append_paged_kv_cache_kernel does not support k_head_dim being different from v_head_dim
#877 opened Feb 19, 2025 by qiyuxinlin
2
Prebuilt kernels not found, using JIT backend
#876 opened Feb 19, 2025 by rangehow
6
BatchPrefillWithRaggedKVCacheWrapper custom_mask doesn't work for small batch sizes when head_dim_qk != head_dim_vo
#860 opened Feb 17, 2025 by pankajroark
@yzh119
1
[Feature] Can BatchDecodeWithPagedKVCacheWrapper return attention scores to all tokens, not just logsumexp?
#838 opened Feb 14, 2025 by yawnzh
1
PrefillPlan tries to allocate more memory than float_workspace_size_in_bytes passed in.
#809 opened Feb 12, 2025 by rchardx
1
[BUG] attention/prefill.cuh(138): error: expression must have a constant value
#806 opened Feb 11, 2025 by haohaibo
4
Numerical stability issue in recent commits since 0.2.0 bug Something isn't working priority: high
#805 opened Feb 11, 2025 by rchardx
5
[Docs] TypeError: single_prefill_with_kv_cache() got an unexpected keyword argument 'use_fp16_qk_reduction'
#802 opened Feb 10, 2025 by sleepwalker2017
@yzh119
[Feature] Reuse JIT code path for building AOT wheel enhancement New feature or request
#791 opened Feb 6, 2025 by abcdabcd987
@abcdabcd987
6
Question about cascade inference
#789 opened Feb 5, 2025 by sleepwalker2017
12
ProTip! What’s not been updated in a month: updated:<2025-02-08.