Skip to content

Commit 2b4cb8a

Browse files
k50112113dllehr-amd
authored andcommitted
fix fp8 kv_cache bug
1 parent 2f1f148 commit 2b4cb8a

File tree

1 file changed

+2
-0
lines changed

1 file changed

+2
-0
lines changed

vllm/v1/attention/backends/rocm_aiter_fa.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -432,6 +432,8 @@ def __init__(
432432
if self.sinks is not None:
433433
raise NotImplementedError("Sinks are not supported for ROCM AITER")
434434

435+
self.fp8_dtype = current_platform.fp8_dtype()
436+
435437
def forward(
436438
self,
437439
layer: torch.nn.Module,

0 commit comments

Comments
 (0)