[MLAS/NEON] Add dedicated kernel for depthwise convolution for ARM64 using NEON intrinsics

[hariharans29]

Description

Motivation and approach taken:

Add a dedicated depthwise convolution kernel for the most common depthwise convolution configuration (3x3 filter, stride = 1, pad <= 1, dilation = 1) using NEON intrinsics. This does significantly better than the current approach of Im2Col + SGemm. The Im2Col step extracts convolution patches and this is a wasteful step and for a 3x3 filter, K would be 9 for the SGemm and usually Gemms are not optimized for such small K values. Hence, a dedicated kernel works much better.

Initially, I ported over the Winograd based NEON accelerated depthwise convolution kernel from PyTorch but I found that its performance is not very good. It's poor performance is probably due to applying the Winograd transformation for the filter repeatedly. A better approach may be to tranform the filter offline and this approach can be considered for later (I reverted the PyTorch Winograd implementation in this commit: 2820a84).

The current depthwise kernel added in this PR was authored by GPT5.1-Codex and with some minor bug fixes it seems to be functionally correct now and also provides the perf boost we are seeking.

Unit tests:
There are already depthwise convolution tests already existing in the codebase. I don't see a need for new ones at this point.

Kernel benchmarking:
This is the kernel level perf improvement from MLAS Conv benchmarks (About 50% kernel latency improvements):

Motivation and Context

A key customer model had a few depthwise conolution operations and this change provides a non-negligible ~3% throughput improvement using the customer provided benchmarking setup

For those interested, #26654 adds support for the same type of convolution variant but that leverages SME1/SME2 through KleidiAI. This PR is conceptually the same but targeting NEON only platforms.

[hariharans29]

/azp run Linux QNN CI Pipeline,Win_TRT_Minimal_CUDA_Test_CI,Windows ARM64 QNN CI Pipeline,Windows GPU Doc Gen CI Pipeline

[azure-pipelines]

Azure Pipelines successfully started running 4 pipeline(s).

[github-actions]

You can commit the suggested changes from lintrunner.

[yuslepukhin]

[Rohanjames1997]

Since this PR and #26838 were merged in a short time window, we may have hit an edge case where the CI for #26838 succeeded, but it was last run before this PR merged.*

So, we still have a reference to sconv.h lying around here which is failing the mainline build when run with --enable_arm_neon_nchwc

onnxruntime/onnxruntime/core/mlas/lib/sbconv_kernel_neon.cpp

Line 20 in 01f40e6

#include "sconv.h"

*This is my hypothesis. I do not know if the CI is extensive enough to cover --enable_arm_neon_nchwc

[hariharans29]

Since this PR and #26838 were merged in a short time window, we may have hit an edge case where the CI for #26838 succeeded, but it was last run before this PR merged.*

So, we still have a reference to sconv.h lying around here which is failing the mainline build when run with --enable_arm_neon_nchwc

onnxruntime/onnxruntime/core/mlas/lib/sbconv_kernel_neon.cpp

Line 20 in 01f40e6

#include "sconv.h"

*This is my hypothesis. I do not know if the CI is extensive enough to cover --enable_arm_neon_nchwc

Oh ! I think the reason is NCHWC is only enabled on MacOS now. With th BF16 feature being turned on only on Linux, I think we may have to enable it in the Linux CI as well. We are a bit busy with the 1.24 release and I will add it to the backlog and re-visit later.

Can you please propose a fix ? FWIW - This is a PR that might interest you AND have the fix - #27099. The PR may not go in 1.24 at this point and so if you could please provide an isolated fix - I will ensure the isolated fix gets cherry-picked into 1.24. Thanks!

[Rohanjames1997]

Cool np!

Yeah, I did see #27099 and it prompted me to check why it wasn't caught earlier.

I will raise a fix for it now.

[Rohanjames1997]

#27101