Problem
The current estimator is useful, but still too generic for some real workloads. It treats long context mostly as a linear KV-cache cost and does not model things like sliding-window attention, KV cache quantization, batch parallelism, MTP, or backend-specific behavior such as MLX vs llama.cpp vs Vulkan/ROCm.
Done when
- The estimator has explicit inputs or model metadata for at least sliding-window attention and KV cache quantization.
- Long-context VRAM estimates change according to model architecture instead of one global formula.
- Backend/runtime differences are represented clearly enough that Apple MLX, llama.cpp, Vulkan, ROCm, and CUDA do not all collapse into one rough vendor multiplier.
- Existing estimates remain conservative when metadata is unknown.
- Tests cover dense attention, sliding-window attention, and at least one backend-specific speed path.
Problem
The current estimator is useful, but still too generic for some real workloads. It treats long context mostly as a linear KV-cache cost and does not model things like sliding-window attention, KV cache quantization, batch parallelism, MTP, or backend-specific behavior such as MLX vs llama.cpp vs Vulkan/ROCm.
Done when