fix(platform): stub torch.distributed._symmetric_memory for torch < 2.8 - #434
Open
tengqm wants to merge 1 commit into
Open
fix(platform): stub torch.distributed._symmetric_memory for torch < 2.8#434tengqm wants to merge 1 commit into
tengqm wants to merge 1 commit into
Conversation
vllm.distributed.parallel_state imports torch.distributed._symmetric_memory at module level, but the module only exists in PyTorch 2.8+. Vendor torch builds on 2.7.x (iluvatar corex) therefore crash with ImportError before engine start. Pre-register an empty stub (mirrored on the parent package) so the import gate passes; all of vllm's runtime uses of the module are lazy.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
vllm.distributed.parallel_stateimportstorch.distributed._symmetric_memoryat module level (line 42), but that module only exists in PyTorch 2.8+.
Vendor torch builds on 2.7.x — iluvatar corex ships torch 2.7.1 — therefore
crash with
ImportErrorbefore the engine core starts, in both the API-serverand spawned worker processes.
This PR pre-registers an empty stub module (plus a mirror of the name on the
torch.distributedparent package, since a baresys.modulesentry does notset the parent attribute) whenever
torch.distributed._symmetric_memoryisabsent. All of vllm's runtime uses of the module are lazy (inside functions),
so the stub only needs to satisfy the module-level import gate; it is never
consulted on the verified greedy / sampling paths. The capability check keeps
PyTorch 2.8+ builds untouched.
This mirrors the existing module-top sentinel pattern already used for
torch.float4_e2m1fn_x2(vllm_fl/__init__.py).Notes
app image, serve-gate crash at
vllm/distributed/parallel_state.py:42.A container-level
try/exceptaround the import made serve fully green(greedy decode and temperature>0 sampling both clean).
module, and the stub is a no-op where the module exists.
Verification
Pending on-node: iluvatar-corex4.4.0 app image serve E2E with Qwen3-4B,
built from the PR-head wheel.
This PR was written in part with the assistance of generative AI.