-
Notifications
You must be signed in to change notification settings - Fork 3.6k
[QNN-EP] Disable DSPQ polling when needed #26468
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
[QNN-EP] Disable DSPQ polling when needed #26468
Conversation
73c1590 to
1cc8bbc
Compare
|
This PR is based on another PR: The reason for this is that PR 26457 removes per-thread HTP power config ids and creates these ids on a per-session basis. Implementing this PR's changes without PR 26457's changes would require unnecessary work that would otherwise be removed by PR 26457 anyway. |
537f206 to
1eac4d5
Compare
- If performance mode updates are not burst, disable DSPQ polling
1eac4d5 to
1515f4a
Compare
| default_rpc_polling_time_, | ||
| default_rpc_control_latency_); | ||
|
|
||
| if (rt != Status::OK()) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is missing negation. !rt.IsOK()
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR introduces a new HtpPowerConfigManager class to handle HTP power configuration updates more efficiently and fixes an issue where DSPQ polling remained enabled after switching from burst mode to other performance modes. The key improvement is that power configs are only updated when values actually change, and RPC polling is properly disabled (set to 0) when performance mode is not burst.
Key changes:
- Created
HtpPowerConfigManagerclass to track and stage power configuration changes before applying them - Modified power config logic to explicitly disable DSPQ polling (rpc_polling_time=0) when performance mode is not burst
- Separated
kHtpSustainedHighPerformancefromkHtpBurstwith different voltage corner settings
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
qnn_htp_power_config_manager.h |
New header defining the HtpPowerConfigManager class that stages and applies power config changes |
qnn_htp_power_config_manager.cc |
Implementation of HtpPowerConfigManager with logic to track last-set values and only update changed configs |
qnn_backend_manager.h |
Updated to use the new HtpPowerConfigManager and consolidated SetHtpPowerConfigs method |
qnn_backend_manager.cc |
Refactored to use HtpPowerConfigManager, removing old SetHtpPowerConfig and SetRpcPowerConfigs methods |
qnn_execution_provider.h |
Added SetHtpPowerConfigs declaration and changed GetPerThreadHtpPowerConfigs signature to return bool |
qnn_execution_provider.cc |
Updated logic to set rpc_polling_time based on performance mode (9999 for burst, 0 otherwise) |
qnn_def.h |
Updated PerThreadHtpPowerConfigs struct to use separate optionals for RPC configs and added new constants |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
onnxruntime/core/providers/qnn/builder/qnn_htp_power_config_manager.cc
Outdated
Show resolved
Hide resolved
onnxruntime/core/providers/qnn/builder/qnn_htp_power_config_manager.cc
Outdated
Show resolved
Hide resolved
onnxruntime/core/providers/qnn/builder/qnn_htp_power_config_manager.cc
Outdated
Show resolved
Hide resolved
onnxruntime/core/providers/qnn/builder/qnn_htp_power_config_manager.cc
Outdated
Show resolved
Hide resolved
|
/azp run Linux QNN CI Pipeline,Win_TRT_Minimal_CUDA_Test_CI,Windows ARM64 QNN CI Pipeline,Windows GPU Doc Gen CI Pipeline |
|
Azure Pipelines successfully started running 4 pipeline(s). |
yuslepukhin
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🕐
Description
Create new class to handle HTP power config updates
Only update if there are changes to power config settings
On dynamic htp perf mode updates, disable DSPQ polling if perf mode is not burst
Motivation and Context
Currently, if a session has set the performance mode to burst then changed the performance mode to anything else, DSPQ polling will be enabled and never disabled. This change is to allow disabling of DSPQ polling when the performance mode is not burst, even on updates.