Skip to content

feat: support FlashComm1 sequence parallelism for DeepSeekV4 NPU. - #2022

Open
chenchuw886 wants to merge 3 commits into
xLLM-AI:mainfrom
chenchuw886:flashcomm1-clean
Open

feat: support FlashComm1 sequence parallelism for DeepSeekV4 NPU.#2022
chenchuw886 wants to merge 3 commits into
xLLM-AI:mainfrom
chenchuw886:flashcomm1-clean

Conversation

@chenchuw886

@chenchuw886 chenchuw886 commented Jul 23, 2026

Copy link
Copy Markdown

Summary

This PR extends the NPU FlashComm1 sequence-parallel path introduced in
#1890 to DeepSeek V4 /
DSV4-Flash.

During prefill, hidden states are sharded across the TP group and kept in the
sharded layout where supported. The full-sequence layout is restored before
DeepSeek Sparse Attention and routed-MoE computation, then converted back to
the sharded layout through ReduceScatter or explicit re-sharding. The final
hidden states are gathered before the model head and MTP state handling.

FlashComm1 remains disabled by default:

--enable_flashcomm1=false

Enable it with:

--enable_flashcomm1=true \
--flashcomm1_min_prefill_tokens=1000

MMRS fusion remains an independent opt-in inherited from #1890:

--enable_mmrs_fusion=true \
--mmrs_comm_mode=aiv

The supplied performance record identifies only the FlashComm1 off/on toggle;
it does not identify the MMRS setting or communication mode. Therefore, the
results below make no separate MMRS performance claim.

Main Changes

  • Integrate FlashComm1Context into the DeepSeek V4 NPU prefill path.
  • Shard model hidden states before the decoder layers and gather them before
    model-output and MTP state handling.
  • Restore the full-sequence layout before DeepSeek Sparse Attention and
    routed-MoE computation.
  • Use the FlashComm1 row-parallel reduce mode for the DeepSeek Sparse Attention
    output projection.
  • Re-shard routed-MoE output before returning to the sequence-parallel decoder
    layout.
  • Exclude the shared-expert DenseMLP from the generic FlashComm1
    gather/reduce-scatter path, avoiding an incorrect gather inside the
    shared-expert branch.
  • Remove the Qwen3.5-specific dp=1 and tp>=8 activation restrictions so the
    tested dp4 tp4 topology can use FlashComm1.
  • Change the global default prefill-token threshold from 8192 to 1000. This
    policy change also applies to the existing Qwen3.5 FlashComm1 path.
  • Add unit coverage for eligibility, context scope, sequence sharding, tail
    padding, and reduce-mode selection.

Default Activation Policy

FlashComm1 becomes active only when all applicable runtime conditions are met:

  • NPU backend.
  • --enable_flashcomm1=true.
  • Prefill-only forward.
  • Current prefill token count is at least
    --flashcomm1_min_prefill_tokens, which now defaults to 1000.
  • cp_size=1; context parallelism remains unsupported.
  • A TP process group is available. More than one TP rank is required for
    actual sequence sharding.
  • The current DeepSeek V4 forward is not an ACL Graph forward.
  • The current DP rank is not empty.

There is no longer an explicit dp=1 or tp>=8 requirement. Decode remains
outside the FlashComm1 path.

ACL Graph was enabled in the benchmark deployment, but the implementation does
not run FlashComm1 inside an ACL Graph forward. FlashComm1 operates on the
eligible non-graph prefill path, while graph execution can remain enabled for
the deployment.

Validation

Unit coverage added

  • FlashComm1 eligibility for dp4 tp4.
  • Minimum-prefill-token and prefill-only gating.
  • Rejection when context parallelism is enabled.
  • Thread-local context publication and restoration.
  • Divisible sequence sharding.
  • Tail-rank-only padding for uneven token counts.
  • Row padding and zero-fill behavior.
  • ReduceScatter versus MMRS reduce-mode selection.

Checks performed

  • pre-commit run --all-files: passed.
  • Performance benchmark: completed with the supplied results below.

Performance

Test Configuration

Item Value
Model DSV4-Flash
Precision W8A8C16
Device Ascend A3
Deployment A3 single-node mixed deployment, EP16
Test tool ais-bench
Dataset gsm8k
Input / output 8K / 1K
Parallel configuration dp4 tp4
Concurrency 32 and 1
Prefix cache off
MTP on
ACL Graph on
Async scheduling on
Chunked Prefill off
PCP (cp_size) off

Measured Results

Concurrency FlashComm1 Avg TTFT (ms) Avg TPOT (ms) Total throughput / card Output throughput / card Input throughput / card
32 off 10211.20 49.60 604.57 67.15 537.42
32 on 9003.80 46.90 646.43 71.80 574.62
1 off 1002.60 21.30 50.43 5.60 44.83
1 on 882.80 21.30 50.76 5.64 45.12
Concurrency TTFT improvement TPOT improvement
32 11.82% 5.44%
1 11.95% 0.00%

At concurrency 32, total throughput per card increases from 604.57 to 646.43.
At concurrency 1, it increases from 50.43 to 50.76.

@DongheJin

DongheJin commented Jul 27, 2026

Copy link
Copy Markdown
Collaborator

Commit messages must conform to the specifications and must end with a dot.

@chenchuw886
chenchuw886 force-pushed the flashcomm1-clean branch 2 times, most recently from 68a6b53 to 464e245 Compare July 28, 2026 01:32
@chenchuw886

Copy link
Copy Markdown
Author

Commit messages must conform to the specifications and must end with a dot.

Done.

@chenchuw886
chenchuw886 marked this pull request as ready for review July 28, 2026 05:00
@chenchuw886 chenchuw886 changed the title feat: add flashcomm1 feature for deepseek v4 on NPU feat: support FlashComm1 sequence parallelism for DeepSeekV4 NPU. Jul 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants