Skip to content

test(e2e): run the responses suite on vllm and tokenspeed #2044

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
slin1237 wants to merge 1 commit into
base: main
Choose a base branch
from
Open

test(e2e): run the responses suite on vllm and tokenspeed #2044

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 21 additions & 3 deletions .github/workflows/pr-test-rust.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -655,6 +655,7 @@ jobs:
# and run under e2e-1gpu-chat above.

e2e-1gpu-responses:
name: e2e-1gpu-responses (${{ matrix.engine }})
needs: [build-wheel, detect-changes]
if: >-
always()
Expand All @@ -664,13 +665,30 @@ jobs:
|| (needs.detect-changes.result == 'success'
&& (needs.detect-changes.outputs.common == 'true'
|| needs.detect-changes.outputs.agentic == 'true')))
strategy:
fail-fast: false
matrix:
include:
- engine: sglang
timeout: 28
test_timeout: 20
- engine: vllm
timeout: 28
test_timeout: 20
# tokenspeed builds from source (~30m cold), so keep the job
# timeout generous even though the test step is short. Only the
# cross-engine gpt-oss-20b responses tests select onto this lane;
# the engine marker filter deselects the sglang-only cases.
- engine: tokenspeed
timeout: 50
Comment on lines +671 to +683

@claude claude Bot Aug 4, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Nit: Five response tests are marked @pytest.mark.engine("sglang", "vllm", "trtllm", "tokenspeed") but the matrix only adds vllm and tokenspeed — trtllm is still missing. The chat lane already runs trtllm on the same 1-gpu-h100 runner, so adding it here would close the remaining coverage gap. Intentional omission or follow-up?

test_timeout: 20
uses: ./.github/workflows/e2e-gpu-job.yml
with:
engine: sglang
engine: ${{ matrix.engine }}
gpu_tier: "1"
runner: 1-gpu-h100
timeout: 28
test_timeout: 20
timeout: ${{ matrix.timeout }}
test_timeout: ${{ matrix.test_timeout }}
test_dirs: e2e_test/responses
setup_agentic_deps: true
secrets: inherit
Expand Down
Loading