Skip to content

[Pytorch] MegaMOE Fused Op Integration - #3350

Open
vthumbe1503 wants to merge 149 commits into
NVIDIA:mainfrom
vthumbe1503:dispatch_combine_op
Open

vthumbe1503 wants to merge 149 commits into
NVIDIA:mainfrom
vthumbe1503:dispatch_combine_op

Conversation

@vthumbe1503

Copy link
Copy Markdown
Collaborator

Description

Please include a brief summary of the changes, relevant motivation and context.

Fixes # (issue)

Type of change

  • Documentation change (change only to the documentation, either a fix or a new content)
  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Infra/Build change
  • Code refactoring

Changes

Please list the changes introduced in this PR:

  • Change A
  • Change B

Checklist:

  • I have read and followed the contributing guidelines
  • The functionality is complete
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

vthumbe1503 and others added 30 commits July 28, 2026 23:05
Signed-off-by: Varun Thumbe <vthumbe@nvidia.com>
…h error handling tests

Signed-off-by: Varun Thumbe <vthumbe@nvidia.com>
Signed-off-by: Varun Thumbe <vthumbe@nvidia.com>
Signed-off-by: Varun Thumbe <vthumbe@nvidia.com>
Signed-off-by: Varun Thumbe <vthumbe@nvidia.com>
Signed-off-by: Varun Thumbe <vthumbe@nvidia.com>
Signed-off-by: Varun Thumbe <vthumbe@nvidia.com>
Signed-off-by: Varun Thumbe <vthumbe@nvidia.com>
Signed-off-by: Varun Thumbe <vthumbe@nvidia.com>
Signed-off-by: Varun Thumbe <vthumbe@nvidia.com>
Signed-off-by: Varun Thumbe <vthumbe@nvidia.com>
Signed-off-by: Varun Thumbe <vthumbe@nvidia.com>
Signed-off-by: Varun Thumbe <vthumbe@nvidia.com>
Signed-off-by: Varun Thumbe <vthumbe@nvidia.com>
Signed-off-by: Varun Thumbe <vthumbe@nvidia.com>
Signed-off-by: Varun Thumbe <vthumbe@nvidia.com>
Signed-off-by: Varun Thumbe <vthumbe@nvidia.com>
Signed-off-by: Varun Thumbe <vthumbe@nvidia.com>
Signed-off-by: Varun Thumbe <vthumbe@nvidia.com>
Signed-off-by: Varun Thumbe <vthumbe@nvidia.com>
Signed-off-by: Varun Thumbe <vthumbe@nvidia.com>
Signed-off-by: Varun Thumbe <vthumbe@nvidia.com>
…/TransformerEngine into enable_extra_out_consumption
Signed-off-by: vthumbe <vthumbe@nvidia.com>
Signed-off-by: Varun Thumbe <vthumbe@nvidia.com>
Signed-off-by: vthumbe <vthumbe@nvidia.com>
Signed-off-by: vthumbe <vthumbe@nvidia.com>
Signed-off-by: vthumbe <vthumbe@nvidia.com>
Signed-off-by: vthumbe <vthumbe@nvidia.com>
Signed-off-by: vthumbe <vthumbe@nvidia.com>
@vthumbe1503 vthumbe1503 changed the title [Pytorch] MOE Sequential Block [Pytorch] MegaMOE Fused Op Integration Sep 14, 2026
vthumbe1503 and others added 2 commits September 17, 2026 13:45
Signed-off-by: vthumbe <vthumbe@nvidia.com>
vthumbe1503 and others added 4 commits September 17, 2026 22:00
Signed-off-by: root <root@hecate0234.hecate.clusters.nvidia.com>
Signed-off-by: Varun Thumbe <vthumbe@nvidia.com>
Signed-off-by: vthumbe <vthumbe1503>
@vthumbe1503 vthumbe1503 added 2.21 and removed 2.20 labels Sep 18, 2026
Signed-off-by: Varun Thumbe <vthumbe@nvidia.com>
Signed-off-by: Varun Thumbe <vthumbe@nvidia.com>
@greptile-apps

greptile-apps Bot commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Comments Outside Diff

These findings could not be posted inline.

  • P2 Payload Errors Can Pass tests/pytorch/distributed/run_ep.py:548 ▶

    This comparison sums each received token’s features before sorting the rows. Swapped or compensating feature errors can leave those sums unchanged, so the MXFP8 dispatch test can pass with an incorrect token payload. Comparing complete rows in an order-independent way would preserve the test’s ability to detect those errors.

    Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Signed-off-by: vthumbe1503 <vthumbe@nvidia.com>
Signed-off-by: vthumbe1503 <vthumbe@nvidia.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants