Skip to content

Commit 002d388

Browse files
philippebichclaude
andcommitted
docs: README note now lists supported page sizes 64 and 128
The tile / page size note said it was fixed at 128 with other sizes coming soon. Block size 64 is now supported (kvarn_k4v2_g64 / kvarn_k4v4_g64), so the note lists both 64 and 128 and the granularity-vs-capacity tradeoff. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 42899a5 commit 002d388

1 file changed

Lines changed: 6 additions & 2 deletions

File tree

README.md

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -78,8 +78,12 @@ Serving works the same way:
7878
vllm serve Qwen/Qwen3-32B --dtype float16 --kv-cache-dtype kvarn_k4v2_g128 --block-size 128
7979
```
8080

81-
> **Note:** KVarN runs in `float16` compute. The tile / page size is currently
82-
> fixed at 128 (one vLLM block = one KVarN tile); other page sizes are coming soon.
81+
> **Note:** KVarN runs in `float16` compute. One vLLM block is one KVarN tile, so
82+
> the tile / page size equals `--block-size`. Both **128** (default) and **64** are
83+
> supported, selected by the matching preset (`kvarn_k4v2_g128` / `kvarn_k4v2_g64`).
84+
> 128 is the design point; 64 gives finer quantization granularity at the cost of a
85+
> little KV capacity (more per-tile scale overhead per token), at essentially the
86+
> same throughput.
8387
8488
> **Tip (capacity):** KVarN realizes its full KV-cache capacity when there is room
8589
> to amortize a small fixed decode workspace. On multi-GPU or generous

0 commit comments

Comments
 (0)