Skip to content

docs(model-deployment): record the vLLM-Ascend EPD row as verified on hardware #767

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
thxCode merged 2 commits into from
Oct 18, 2026
Merged

docs(model-deployment): record the vLLM-Ascend EPD row as verified on hardware #767

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
0af9986
docs(model-deployment): record the vLLM-Ascend EPD row as verified on…
thxCode Oct 18, 2026
c554875
fixup! docs(model-deployment): record the vLLM-Ascend EPD row as veri…
thxCode Oct 18, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions docs/modules/model-deployment/engine-versions.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ those releases, and this documentation does not track them; upgrading is the fix
| SGLang | prefill/decode, direct transfer over `tcp` | `0.5.18` | `0.3.12.post1` | — |
| SGLang | with a Mooncake store | `0.5.18`, run as a pair | `0.3.12.post1` | the 0.3.12 line, such as `kvcacheai/mooncake:0.3.12.post1` |
| vLLM | encoder/prefill/decode, direct transfer over NIXL (`ECCPUConnector`) | `0.30.0`, behind `llm-d-router` or `vllm-router`: NVIDIA CUDA (verified on 4 × NVIDIA L40S) or AMD ROCm (unverified on hardware) | compiles its own, not run | — |
| vLLM-Ascend | encoder/prefill/decode, direct transfer (`ECExampleConnector`) | `0.30.0rc1`, behind `llm-d-router` (unverified on hardware) | `0.3.11.post1`, the NPU build | — |
| vLLM-Ascend | encoder/prefill/decode, direct transfer (`ECExampleConnector`) | `0.30.0rc1`, behind `llm-d-router` (verified on Ascend 910B2) | `0.3.11.post1`, the NPU build | — |
| SGLang | encoder/prefill/decode, transfer by address (`encoder-urls`) | `0.5.10`, behind `llm-d-router` or `sglang-gateway` (NVIDIA verified on 2 × NVIDIA L40S) | `0.3.12.post1` | — |

A version counts as run when the shape answered requests and, where it moves KV, the engine or the
Expand All @@ -33,8 +33,9 @@ vLLM-Ascend rows ran on the runner's `cann9.1-910b-vllm0.23.0` tag, named throug
`roles[].image`.

The NVIDIA vLLM EPD rows (behind `llm-d-router` and `vllm-router`) and the NVIDIA SGLang EPD rows
(run on `0.5.18`, behind both routers) ran live. The AMD and Ascend EPD rows are admitted but not
verified on hardware ([#763](https://github.com/gpustack/gpustack-operator/issues/763) tracks the SGLang ones).
(run on `0.5.18`, behind both routers) ran live. The vLLM-Ascend EPD row behind `llm-d-router` ran live on Ascend 910B2. The AMD EPD rows and the
SGLang Ascend rows are admitted but not verified on hardware
([#763](https://github.com/gpustack/gpustack-operator/issues/763) tracks them).

## Reading the table

Expand Down
4 changes: 2 additions & 2 deletions docs/modules/model-deployment/epd.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ Admission accepts only the combinations below. Anything else is refused when the
|---|---|---|---|---|---|
| `llm-d-router` | `vLLM` | `nvidia` | `ECCPUConnector` over NIXL | `0.30.0` | Verified on 4 × NVIDIA L40S |
| `llm-d-router` | `vLLM` | `amd` (ROCm) | `ECCPUConnector` over NIXL | `0.30.0` | Not verified |
| `llm-d-router` | `vLLM` | `ascend` | `ECExampleConnector` over a shared volume | `0.30.0rc1` | Not verified |
| `llm-d-router` | `vLLM` | `ascend` | `ECExampleConnector` over a shared volume | `0.30.0rc1` | Verified on Ascend 910B2 |
| `vllm-router` | `vLLM` | `nvidia` | `ECCPUConnector` over NIXL | `0.30.0` | Verified on 4 × NVIDIA L40S |
| `vllm-router` | `vLLM` | `amd` (ROCm) | `ECCPUConnector` over NIXL | `0.30.0` | Not verified |
| `llm-d-router` | `SGLang` | `nvidia` | `encoder-urls` | `0.5.10` | Verified on 2 × NVIDIA L40S |
Expand All @@ -69,7 +69,7 @@ Admission accepts only the combinations below. Anything else is refused when the

A verified row served a multimodal request through the router on the hardware named. A row marked
Not verified is admitted from the design and has not run on that hardware. Issue
[#763](https://github.com/gpustack/gpustack-operator/issues/763) tracks the SGLang AMD and Ascend rows.
[#763](https://github.com/gpustack/gpustack-operator/issues/763) tracks the rows still unrun: the AMD ROCm rows and the SGLang Ascend rows.
The `Minimum version` column is a requirement on the image you name, not an admission check:
nothing reads a version, so an image below it is applied as written.

Expand Down
Loading