fix(kvarn): avoid double-counting weights when a repo ships both consolidated and sharded safetensors

[sztlink]

Summary

KVarNConfig.estimate_weight_bytes globs and sums every *.safetensors in the model snapshot. Repos that ship a single consolidated checkpoint alongside the sharded HF set get counted at roughly 2x their real weight size. The clearest case is mistralai/Mistral-7B-Instruct-v0.3, whose snapshot carries consolidated.safetensors (the original Mistral format) next to model-0000n-of-0000m.safetensors (the HF shards).

The over-estimate propagates through the pool sizing: the post-weight usable envelope gpu_memory_utilization * total - weight_bytes goes negative, pool_budget_bytes floors at 0, and check_and_update_config caps max_num_seqs to 1. KVarN then serves the model at single-stream throughput only (the log shows capping max_num_seqs 256 -> 1), while fp16 and other KV-quant backends serve it normally. vLLM itself loads the model correctly because it reads model.safetensors.index.json; only this estimate helper double-counts.

Reproduction

vllm serve mistralai/Mistral-7B-Instruct-v0.3 \
  --kv-cache-dtype kvarn_k4v2_g128 --block-size 128 \
  --gpu-memory-utilization 0.80 --max-model-len 4096 --enforce-eager

Log on main:

KVarN (kvarn_k4v2_g128): capping max_num_seqs 256 -> 1 so the fp16 tail pool fits its budget ...
Reserved 0.66 GiB for the KVarN fp16 tail pool.

Fix

Prefer the *.safetensors.index.json (or *.bin.index.json) manifest and sum only the shards it names, which is exactly the set the loader reads. Fall back to a single canonical checkpoint (model.safetensors, then consolidated.safetensors), then to the previous glob. No behavior change for models that ship only the sharded set or a single checkpoint.

Before / after (measured, 24 GB GPU, gmu 0.80)

	estimate_weight_bytes	cap (max_num_seqs)	fp16 tail pool	serves
Mistral-7B-Instruct-v0.3, before	27.00 GB	1	0.66 GiB	single-stream only
Mistral-7B-Instruct-v0.3, after	13.50 GB	71	2.84 GiB	yes (verified generation)

Controls (only sharded set, no consolidated file), estimate unchanged before and after:

model	estimate (before = after)
Qwen2.5-7B-Instruct	14.19 GB
Qwen3-8B	15.26 GB

Validation

mistralai/Mistral-7B-Instruct-v0.3 with kvarn_k4v2_g128 now serves and generates correctly (cap 71, nvidia-smi shows normal concurrent decode). Validated on WSL2 (vLLM editable install), single 24 GB GPU, gpu_memory_utilization 0.80.