Skip to content

fix(platform): stub torch.distributed._symmetric_memory for torch < 2.8 - #434

Open
tengqm wants to merge 1 commit into
flagos-ai:mainfrom
tengqm:fix/iluvatar-symm-mem-stub
Open

fix(platform): stub torch.distributed._symmetric_memory for torch < 2.8#434
tengqm wants to merge 1 commit into
flagos-ai:mainfrom
tengqm:fix/iluvatar-symm-mem-stub

Conversation

@tengqm

@tengqm tengqm commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Summary

vllm.distributed.parallel_state imports torch.distributed._symmetric_memory
at module level (line 42), but that module only exists in PyTorch 2.8+.
Vendor torch builds on 2.7.x — iluvatar corex ships torch 2.7.1 — therefore
crash with ImportError before the engine core starts, in both the API-server
and spawned worker processes.

This PR pre-registers an empty stub module (plus a mirror of the name on the
torch.distributed parent package, since a bare sys.modules entry does not
set the parent attribute) whenever torch.distributed._symmetric_memory is
absent. All of vllm's runtime uses of the module are lazy (inside functions),
so the stub only needs to satisfy the module-level import gate; it is never
consulted on the verified greedy / sampling paths. The capability check keeps
PyTorch 2.8+ builds untouched.

This mirrors the existing module-top sentinel pattern already used for
torch.float4_e2m1fn_x2 (vllm_fl/__init__.py).

Notes

  • Root-caused on-node: iluvatar-corex4.4.0 (torch 2.7.1 + corex), vllm 0.24.0
    app image, serve-gate crash at vllm/distributed/parallel_state.py:42.
    A container-level try/except around the import made serve fully green
    (greedy decode and temperature>0 sampling both clean).
  • Deliberately not gated on a specific vendor: any torch build < 2.8 lacks the
    module, and the stub is a no-op where the module exists.

Verification

Pending on-node: iluvatar-corex4.4.0 app image serve E2E with Qwen3-4B,
built from the PR-head wheel.


This PR was written in part with the assistance of generative AI.

vllm.distributed.parallel_state imports torch.distributed._symmetric_memory
at module level, but the module only exists in PyTorch 2.8+. Vendor torch
builds on 2.7.x (iluvatar corex) therefore crash with ImportError before
engine start. Pre-register an empty stub (mirrored on the parent package)
so the import gate passes; all of vllm's runtime uses of the module are
lazy.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant