Skip to content

chore(frontend): bump vllm rust crates to 60857baa - #1031

Merged
FeathBow merged 1 commit into
pegainfer-project:mainfrom
FeathBow:chore/bump-vllm-crates-60857baa
Sep 3, 2026
Merged

chore(frontend): bump vllm rust crates to 60857baa#1031
FeathBow merged 1 commit into
pegainfer-project:mainfrom
FeathBow:chore/bump-vllm-crates-60857baa

Conversation

@FeathBow

@FeathBow FeathBow commented Sep 3, 2026

Copy link
Copy Markdown
Collaborator

Description

Closes #1007

  • Bump the five pinned vLLM Rust crates from d3e2888c (2026-08-25) to 60857baa (vllm main as of 2026-09-02).
  • Picks up [Perf][Rust Frontend] Count the tokenizer vocabulary once at construction vllm-project/vllm#54449: the tokenizer wrapper counts its vocabulary once at construction instead of cloning the merged vocabulary on every request, which is what frontend: the tokenizer wrapper counts its vocabulary on every request #1007 measured on the serving path.
  • No bridge changes: the EngineCore impersonation compiles unchanged against the new revision. The tokenizer crate's tekken migration drops tiktoken-rs, rubato, ndarray and hound from the lock.
  • Mac pegainfer-sim A/B at 128-in / 64-out / 3200 prompts / c=320, alternated twice: 0-fail both sides; req/s 372.8 / 370.4 (d3e2888c) vs 374.9 / 374.7 (this branch); TPOT p50 12.16 vs 12.15 ms, the sim floor; TTFT p50 8.9 / 8.7 vs 8.5 / 8.6 ms.
  • Tiny-prompt A/B on the 26B-A4B checkpoint (<bos>hi, one token, 5 warm + 30 timed, median end-to-end, main vs branch alternated twice): 50.5 / 52.4 ms on d3e2888c vs 11.3 / 11.5 ms on this branch, the per-request vocabulary clone frontend: the tokenizer wrapper counts its vocabulary on every request #1007 measured (48.2 ms → 12.5 ms with the cached count applied by hand) now gone through the pin.

Dependency Update

Crate Previous revision New revision
vllm-chat d3e2888c 60857baa
vllm-engine-core-client d3e2888c 60857baa
vllm-server d3e2888c 60857baa
vllm-text d3e2888c 60857baa
vllm-tokenizer d3e2888c 60857baa

Notable upstream in the range (417 commits): the tokenizer vocabulary count cached at construction (#54449), the LogprobsTensors wire schema fix (#53939), the SSE streaming hot-path optimization (#51321), truncate_prompt_tokens / truncation_side support (#48584), stop_token_ids validated against the vocabulary (#54196), gRPC LoRA lifecycle control (#52840), the migration to the new tekken crate (#53056), decoded text attributed to tokens (#52910), bounded recursive argument parsers (#54303), and the DeepSeek V4 renderer fixes (#53281, #54854).

Test plan

  • cargo test --release --locked -p pegainfer-frontend --lib — 67 passed
  • cargo test --release --locked -p pegainfer-sim --lib — 6 passed
  • cargo test --release --locked -p pegainfer-sim --test frontend_e2e — 13 passed
  • Mac sim vllm-bench A/B vs the d3e2888c binary, same flags as above — 3200/0 both, throughput and TPOT flat (numbers above); cargo fmt --check and clippy --all-targets -D warnings clean for pegainfer-frontend and pegainfer-sim
  • Single GPU (sm_89, x86_64), Gemma 4 26B-A4B NVFP4: cargo test --release --locked -p pegainfer-frontend --lib — 67 passed; tiny-prompt A/B above — main 50.5 / 52.4 ms, branch 11.3 / 11.5 ms (p90 within 1.3 ms of the medians on every run)

Signed-off-by: Feathbow <feathbow@gmail.com>
@FeathBow
FeathBow merged commit e7a4197 into pegainfer-project:main Sep 3, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

frontend: the tokenizer wrapper counts its vocabulary on every request

1 participant