feat(router): add cache_aware_length policy with long/short pool split - #2253

Open

jokerhaos wants to merge 14 commits into

smg-project:mainfrom

astro-web3:feature/aiv-785-upstream

Commits on Aug 21, 2026

feat(router): add cache_aware_length policy with long/short pool split
jokerhaos
committed
2bbb6cb
test: cover hit-unhealthy fallback, long-pool-unhealthy overflow, and header-override-char-estimate
jokerhaos
committed
9fa69e1
fix: address CodeRabbit review — token-only handling, perf, tests, CLI validation
jokerhaos
committed
8cec459
fix: address remaining CodeRabbit comments — HTTP path tests, Python binding, English comments
jokerhaos
committed
4063ab6
refactor: make cache_aware_length a full superset of cache_aware via NoCacheStrategy
jokerhaos
committed
c685b4f
feat: add --long-prefill-indices CLI param for long/short pool label injection
jokerhaos
committed
039e3e9
fix: address CodeRabbit review — NaN validation, factory config assertions
jokerhaos
committed
522d613
fix: map cache_aware_length in Python policy_from_str + add NaN validation test
jokerhaos
committed
ba3c416
fix: add cache_aware_length CLI params to Python and Go bindings
jokerhaos
committed
ae5843f
test: add Python binding tests for cache_aware_length CLI params
jokerhaos
committed
54cda08
fix(python): complete cache aware length CLI contract

WyattJia
authored and
jokerhaos
committed
29c00d1
fix: pass uncached hint through NoCacheStrategy and snapshot worker load
jokerhaos
committed
669b894
fix: address all CodeRabbit review comments on PR #2253
jokerhaos
committed
332651f
fix: reject non-finite balance_rel_threshold and add NaN regression test
jokerhaos
committed
60aaac7