Skip to content

[QNN-EP] Support alternate Layernorm fusion pattern in QNN preprocess #26060

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
yuslepukhin merged 2 commits into from
Jan 16, 2026
Merged

[QNN-EP] Support alternate Layernorm fusion pattern in QNN preprocess #26060

yuslepukhin merged 2 commits into from
Jan 16, 2026

Conversation

@qti-mattsinc
Copy link
Contributor

@qti-mattsinc qti-mattsinc commented Sep 16, 2025

Description

Small change to allow QNN Preprocess to allow a Mul node (with A=B) instead of a Pow node (with Y=2) for layernorm fusion.

@HectorSVC HectorSVC added the ep:QNN issues related to QNN exeution provider label Sep 16, 2025
@HectorSVC
Copy link
Contributor

HectorSVC commented Sep 16, 2025

/azp run Linux QNN CI Pipeline,Win_TRT_Minimal_CUDA_Test_CI,Windows ARM64 QNN CI Pipeline,Windows GPU Doc Gen CI Pipeline

@azure-pipelines
Copy link

azure-pipelines bot commented Sep 16, 2025

Azure Pipelines successfully started running 4 pipeline(s).

@yuslepukhin yuslepukhin requested a review from Copilot December 17, 2025 20:05
Copilot AI reviewed Dec 17, 2025
View reviewed changes
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends the QNN preprocessor's LayerNorm fusion capability to recognize an alternate pattern where a Mul node (with both inputs being the same tensor) is used instead of a Pow node (with exponent 2.0) for computing the squared values. This is mathematically equivalent since x² = x * x, and some model exporters may generate this pattern.

Key Changes:

  • Added documentation for the Mul-based LayerNorm fusion pattern
  • Extended pattern matching to recognize Mul nodes in place of Pow nodes
  • Added validation logic to ensure Mul nodes have matching inputs

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

yuslepukhin
yuslepukhin approved these changes Jan 7, 2026
View reviewed changes
Copy link
Member

@yuslepukhin yuslepukhin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

@yuslepukhin
Copy link
Member

yuslepukhin commented Jan 12, 2026

Please, rebase off main

@qti-mattsinc qti-mattsinc force-pushed the dev/mattsinc/AISW-148002 branch 2 times, most recently from 1161fcd to ebaa0cd Compare January 12, 2026 23:29
@qti-mattsinc
Copy link
Contributor Author

qti-mattsinc commented Jan 12, 2026

Rebased on top of tip

@yuslepukhin
Copy link
Member

yuslepukhin commented Jan 13, 2026

/azp run Linux QNN CI Pipeline, Win_TRT_Minimal_CUDA_Test_CI, Windows ARM64 QNN CI Pipeline, Windows GPU CUDA CI Pipeline, Windows GPU DML CI Pipeline, Windows GPU Doc Gen CI Pipeline, Windows GPU TensorRT CI Pipeline, Windows OpenVINO CI Pipeline, Windows x64 QNN CI Pipeline

@azure-pipelines
Copy link

azure-pipelines bot commented Jan 13, 2026

Azure Pipelines successfully started running 4 pipeline(s).

@qti-mattsinc
Copy link
Contributor Author

qti-mattsinc commented Jan 15, 2026 •
edited
Loading

The CI error does not seem related to the change based on the logs. Not sure if this is known flaky behavior. @yuslepukhin, maybe worth retriggering it?

@yuslepukhin
Copy link
Member

yuslepukhin commented Jan 15, 2026 •
edited
Loading

Can you rebase/merge from main? #Resolved

@qti-mattsinc qti-mattsinc force-pushed the dev/mattsinc/AISW-148002 branch from ebaa0cd to 591cbe1 Compare January 15, 2026 22:47
@yuslepukhin
Copy link
Member

yuslepukhin commented Jan 15, 2026

Please, refrain from force pushing, this makes things longer.

@qti-mattsinc
Copy link
Contributor Author

qti-mattsinc commented Jan 15, 2026

Can you rebase/merge from main?

Rebased on top of current tip of main (1c02b79)

@qti-mattsinc
Copy link
Contributor Author

qti-mattsinc commented Jan 15, 2026 •
edited
Loading

Please, refrain from force pushing, this makes things longer.

My bad. For future reference, would it have been correct just to merge main into this branch, then? I'm much more familiar with stacked diff workflows (Gerrit), which would have would have preserved CI checks after a force-push to rebase.

@yuslepukhin
Copy link
Member

yuslepukhin commented Jan 15, 2026

Please, refrain from force pushing, this makes things longer.

My bad. For future reference, would it have been correct just to merge main into this branch, then? I'm much more familiar with stacked diff workflows (Gerrit), which would have would have preserved CI checks after a force-push to rebase.

Yes, simply merging from main is the easiest and preferred way.

@yuslepukhin
Copy link
Member

yuslepukhin commented Jan 15, 2026

/azp run Linux QNN CI Pipeline, Win_TRT_Minimal_CUDA_Test_CI, Windows ARM64 QNN CI Pipeline, Windows GPU CUDA CI Pipeline, Windows GPU DML CI Pipeline, Windows GPU Doc Gen CI Pipeline, Windows GPU TensorRT CI Pipeline, Windows OpenVINO CI Pipeline, Windows x64 QNN CI Pipeline

@azure-pipelines
Copy link

azure-pipelines bot commented Jan 15, 2026

Azure Pipelines successfully started running 4 pipeline(s).

@yuslepukhin
Copy link
Member

yuslepukhin commented Jan 15, 2026

/azp run Linux QNN CI Pipeline,Win_TRT_Minimal_CUDA_Test_CI,Windows ARM64 QNN CI Pipeline,Windows GPU Doc Gen CI Pipeline

@azure-pipelines
Copy link

azure-pipelines bot commented Jan 15, 2026

Azure Pipelines successfully started running 4 pipeline(s).

@yuslepukhin
Copy link
Member

yuslepukhin commented Jan 15, 2026

/azp run Windows ARM64 QNN CI Pipeline

@azure-pipelines
Copy link

azure-pipelines bot commented Jan 15, 2026

Azure Pipelines successfully started running 1 pipeline(s).

@yuslepukhin yuslepukhin merged commit e7dfd69 into microsoft:main Jan 16, 2026
90 of 91 checks passed
alex-spacemit pushed a commit to spacemit-com/onnxruntime that referenced this pull request Jan 20, 2026
@qti-mattsinc @alex-spacemit
[QNN-EP] Support alternate Layernorm fusion pattern in QNN preprocess (…
afe1307 
…microsoft#26060)

### Description
Small change to allow QNN Preprocess to allow a Mul node (with A=B)
instead of a Pow node (with Y=2) for layernorm fusion.
tianleiwu pushed a commit that referenced this pull request Jan 21, 2026
@qti-mattsinc @tianleiwu
[QNN-EP] Support alternate Layernorm fusion pattern in QNN preprocess (…
fb53090 
…#26060)

### Description
Small change to allow QNN Preprocess to allow a Mul node (with A=B)
instead of a Pow node (with Y=2) for layernorm fusion.

(cherry picked from commit e7dfd69)
This was referenced Jan 21, 2026
Closed
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yuslepukhin yuslepukhin yuslepukhin approved these changes

Copilot code review Copilot Copilot left review comments

Assignees

No one assigned

Labels

ep:QNN issues related to QNN exeution provider release:1.24.0

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@qti-mattsinc @HectorSVC @yuslepukhin @edgchen1