test(e2e): TokenSpeed Qwen3-ASR audio transcription hard gate - #1909
test(e2e): TokenSpeed Qwen3-ASR audio transcription hard gate#1909slin1237 wants to merge 1 commit into
Conversation
Add an end-to-end test for POST /v1/audio/transcriptions against a
locally-hosted TokenSpeed gRPC worker serving Qwen/Qwen3-ASR-1.7B. This is
the only serving path for the transcription endpoint: the gRPC router's
transcription adapter is Qwen3-ASR only and audio multimodal inputs are
accepted only on TokenSpeed workers (SGLang/vLLM/TRT-LLM reject audio
batches).
The model is wired through the standard setup_backend gRPC fixture with
@pytest.mark.engine("tokenspeed") + @pytest.mark.gpu(1), so it runs in the
existing e2e-1gpu-chat (tokenspeed) lane (the only lane with TokenSpeed
installed) with no CI workflow changes. Requests use raw httpx multipart to
match the gateway's multipart/form-data contract and stay independent of the
OpenAI SDK's streaming/response-format overloads. Reuses the committed
mary_had_lamb_16k.wav fixture from the realtime ASR e2e.
Covers: whole-file json round-trip (200, non-empty text), response_format=text
(plain-text body), streaming rejection (400), and unsupported-language
rejection (400). The exact ASR wording is logged, not asserted, since it
varies by model revision/build.
Signed-off-by: Simo Lin <linsimo.mark@gmail.com>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughAdds a TokenSpeed/Qwen3-ASR-1.7B E2E test module for audio transcription, covering JSON and plain-text responses plus rejection of streaming and unsupported language requests. ChangesTokenSpeed transcription E2E
Estimated code review effort: 3 (Moderate) | ~15 minutes Suggested reviewers: Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a new end-to-end test suite test_transcription_tokenspeed.py to exercise the POST /v1/audio/transcriptions endpoint against a locally-hosted TokenSpeed gRPC worker serving the Qwen3-ASR model. The tests cover successful round-trip transcription, plain-text response formatting, rejection of streaming requests, and rejection of unsupported languages. There are no review comments, and I have no feedback to provide.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9c20396076
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| return httpx.post( | ||
| f"{base_url}/v1/audio/transcriptions", | ||
| files={"file": _audio_part()}, | ||
| data={"model": MODEL, **data}, |
There was a problem hiding this comment.
When ROUTER_LOCAL_MODEL_PATH resolves this model to a local directory, setup_backend launches and registers the worker under that resolved model_path, while this helper always sends the HuggingFace id. The gRPC worker selection path does an exact model-index lookup, so in those CI/local-cache environments the positive transcription requests can return model_not_found instead of exercising ASR; thread the model_path from setup_backend/the model fixture into the form data instead of hard-coding MODEL.
Useful? React with 👍 / 👎.
lightseek-bot
left a comment
lightseek-bot
left a comment
There was a problem hiding this comment.
shall we use TokenSpeed latest main commit
|
Closing — this reinvents the e2e request layer (raw httpx + a bespoke gateway fixture) instead of using the standard api_client/model fixtures every chat_completions test uses. Rewriting against the existing infra. |
2 participants
Description
Problem
PR #1905 added Qwen3-ASR/Omni audio support served via TokenSpeed, but shipped no e2e coverage for the audio transcription path — the
/v1/audio/transcriptionsTokenSpeed adapter and the Qwen3-ASR multimodal pipeline had no integration test.Solution
Add an e2e test that serves
Qwen/Qwen3-ASR-1.7Bvia TokenSpeed and exercises/v1/audio/transcriptionsend to end: JSON transcription (200 + non-emptytext),response_format=text(plain body), streaming rejection (400), and unsupported-language rejection (400). Reuses the existing 16 kHz WAV fixture; ASR wording is logged (not asserted) to stay robust across model revisions.Feasibility note: Qwen3-ASR's realtime arch (
Qwen3ASRRealtimeGeneration) is vLLM-only; TokenSpeed loads the model's nativeqwen3_asrconfig and the servicer advertisesAUDIOforqwen3_asr, so this path is servable today.Changes
e2e_test/chat_completions/test_transcription_tokenspeed.py(new):TestTokenSpeedTranscription, gRPCsetup_backendfixture, markers@engine("tokenspeed") @gpu(1) @model("Qwen/Qwen3-ASR-1.7B") @e2e @slow.Test Plan
Runs in the existing
e2e-1gpu-chat (tokenspeed)CI lane (test_dirs: e2e_test/chat_completions,E2E_ENGINE=tokenspeed, tier-1 GPU) — no CI-lane change needed; the model is pre-downloaded (tp=1). Locally verified:py_compile+ruff+mypyclean (the e2e itself requires the GPU lane to run).Checklist
cargo +nightly fmtpasses (no Rust changes)ruff/mypycleanSummary by CodeRabbit