Skip to content

Commit e5ef4df

Browse files
toncaocpatonn
andauthored
[Kimi-Linear] Correct prefixes and add compatibility to AWQ quants (vllm-project#27834)
Signed-off-by: toncao <[email protected]> Co-authored-by: toncao <[email protected]>
1 parent 3696050 commit e5ef4df

File tree

1 file changed

+2
-1
lines changed

1 file changed

+2
-1
lines changed

vllm/model_executor/models/kimi_linear.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -155,6 +155,7 @@ def __init__(
155155
hidden_act=config.hidden_act,
156156
quant_config=quant_config,
157157
reduce_results=False,
158+
prefix=f"{prefix}.shared_experts",
158159
)
159160

160161
def forward(self, hidden_states: torch.Tensor) -> torch.Tensor:
@@ -340,7 +341,7 @@ def __init__(
340341
self.block_sparse_moe = KimiMoE(
341342
config=config,
342343
quant_config=quant_config,
343-
prefix=f"{prefix}.mlp",
344+
prefix=f"{prefix}.block_sparse_moe",
344345
)
345346
self.mlp = self.block_sparse_moe
346347
else:

0 commit comments

Comments
 (0)