Skip to content

Conversation

@yuhuchua-qti
Copy link
Contributor

Description

This PR updates the tolerance thresholds for the Clip and Transpose tests in QnnHTPBackendTests. The adjustment accounts for minor accuracy differences introduced by the change in default floating-point precision in QNN HTP starting from version 2.35.

Motivation and Context

Since QNN 2.35, the default floating-point precision in QNN HTP has changed from FP32 to FP16. Additionally, the configuration option QNN_HTP_GRAPH_CONFIG_OPTION_PRECISION has been deprecated.
This change in precision can lead to expected accuracy loss, especially in scenarios where graph inputs and outputs are defined as float 32, but internal computations are performed in FP16 (e.g., FP32 → FP16 → FP32 conversions). To accommodate this, the tolerance thresholds for the affected tests have been increased to prevent false negatives due to precision differences.

@microsoft-github-policy-service agree company="Qualcomm"

@yuhuchua-qti yuhuchua-qti changed the title Adjust tolerance for Clip and Transpose tests due to FP16 default in QNN HTP [QNN EP] Adjust tolerance for Clip and Transpose tests due to FP16 default in QNN HTP Nov 5, 2025
@yuhuchua-qti
Copy link
Contributor Author

@microsoft-github-policy-service agree company="Qualcomm"

@edgchen1 edgchen1 added the ep:QNN issues related to QNN exeution provider label Dec 12, 2025
@yuslepukhin yuslepukhin requested a review from Copilot December 16, 2025 18:20
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adjusts tolerance thresholds for Clip and Transpose operator tests in the QNN HTP backend to accommodate accuracy differences introduced by the default FP16 precision in QNN HTP 2.35+. The precision change affects FP32 models where internal computations now use FP16 (FP32→FP16→FP32 conversions), introducing expected minor accuracy loss.

Key Changes:

  • Removes deprecated enable_fp16_precision provider option handling from test helper functions
  • Increases tolerance from 1e-5f to 5e-3f for affected float32 tests
  • Re-enables previously disabled tests that were failing due to the precision change

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
onnxruntime/test/providers/qnn/transpose_htp_test.cc Refactored RunTransposeNonQDQOnHTP to accept tolerance parameter; re-enabled and renamed TransposeFloat32OnHTP test with adjusted 5e-3f tolerance
onnxruntime/test/providers/qnn/clip_op_test.cc Refactored RunClipTest to accept tolerance parameter; re-enabled Clip_f32 test with 5e-3f tolerance; cleaned up GPU test call to use default parameters

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@yuslepukhin
Copy link
Member

/azp run Linux QNN CI Pipeline, Win_TRT_Minimal_CUDA_Test_CI, Windows ARM64 QNN CI Pipeline, Windows GPU CUDA CI Pipeline, Windows GPU DML CI Pipeline, Windows GPU Doc Gen CI Pipeline, Windows GPU TensorRT CI Pipeline, Windows OpenVINO CI Pipeline, Windows x64 QNN CI Pipeline

@azure-pipelines
Copy link

Azure Pipelines successfully started running 4 pipeline(s).

@yuslepukhin
Copy link
Member

/azp run Linux QNN CI Pipeline, Win_TRT_Minimal_CUDA_Test_CI, Windows ARM64 QNN CI Pipeline, Windows GPU CUDA CI Pipeline, Windows GPU DML CI Pipeline, Windows GPU Doc Gen CI Pipeline, Windows GPU TensorRT CI Pipeline, Windows OpenVINO CI Pipeline, Windows x64 QNN CI Pipeline

@azure-pipelines
Copy link

Azure Pipelines successfully started running 4 pipeline(s).

@edgchen1
Copy link
Contributor

closing and reopening to restart CI pipelines

@edgchen1 edgchen1 closed this Dec 17, 2025
@edgchen1 edgchen1 reopened this Dec 17, 2025
@yuslepukhin
Copy link
Member

Please, address and/or comment on the Copilot review comments so we can get this moving.

* Updated tolerance thresholds to align with FP16 becoming the default precision in QNN HTP.
@yuhuchua-qti yuhuchua-qti force-pushed the dev/yuhuchua/Fix_QnnHTP_UT_for_Clip_and_Transpose_in_FP32_mode branch from bc52f0a to 6e01c0e Compare January 7, 2026 01:52
@yuhuchua-qti
Copy link
Contributor Author

Hi @yuslepukhin
I've addressed Copilot's comment.
Thanks.

@yuslepukhin
Copy link
Member

/azp run Windows CPU CI Pipeline, Windows GPU CI Pipeline, Windows GPU TensorRT CI Pipeline, ONNX Runtime Web CI Pipeline, Windows ARM64 QNN CI Pipeline, Windows x64 QNN CI Pipeline, onnxruntime-python-checks-ci-pipeline

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@yuslepukhin
Copy link
Member

/azp run Linux QNN CI Pipeline, Win_TRT_Minimal_CUDA_Test_CI, Windows ARM64 QNN CI Pipeline, Windows GPU CUDA CI Pipeline, Windows GPU DML CI Pipeline, Windows GPU Doc Gen CI Pipeline, Windows GPU TensorRT CI Pipeline, Windows OpenVINO CI Pipeline, Windows x64 QNN CI Pipeline

@azure-pipelines
Copy link

Azure Pipelines successfully started running 4 pipeline(s).

@yuslepukhin
Copy link
Member

/azp run Linux QNN CI Pipeline, Win_TRT_Minimal_CUDA_Test_CI,Windows ARM64 QNN CI Pipeline,Windows GPU Doc Gen CI Pipeline

@azure-pipelines
Copy link

Azure Pipelines successfully started running 4 pipeline(s).

@yuslepukhin
Copy link
Member

You are force pushing which makes it impossible to focus on your recent changes. Which in turn makes it hard to find out why CI pipelines are failing. Any particular reason why you do not want to preserve the history at review time?

@yuhuchua-qti
Copy link
Contributor Author

Hi @yuslepukhin,
Thanks for pointing this out. I didn’t have a particular reason for force‑pushing — it was just out of habit.
I’ll avoid force‑pushing going forward and make sure to preserve history during review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ep:QNN issues related to QNN exeution provider

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants