Implement new experimental lookup-based matrix multiplication method(TMAC)

[vraspar]

Description

This PR introduces a new experimental lookup-table(LUT) based matrix multiplication method for 2-bit MatMulNBits on x64 AVX2 inspired from T-MAC paper and T-MAC repository to speed up low bit LLM inference.

Unlike the existing quant-dequant methods, the LUT-based method directly supports mixed-precision-GEMM without dequantization. It uses bit-wise table lookup to eliminate multiplications and reduce additions required in matrix multiplication.

This PR:

• Add mlas.use_lut_gemm session option allowing use of LUT GEMM inside matmulnbits when it is available (2-bit, BlkLen multiple of 32, K multiple of 32, N multiple of 128, AVX2 present).
• Introduces LUT packing + kernel config cache (packs bitplanes, scales, ZP) and the main MlasLUTGemm entry that generates per-row LUTs and calls the AVX2 kernel.
• Implements AVX2 LUT generation GenerateLUT_avx2 and GEMM compute TMACComputeGemm_avx2 and wires dispatch in MLAS platform init.
• Updates MatMulNBits PrePack/Compute to use LUT packing/compute when opted-in; keeps existing quant-dequant path as fallback.
• Extends Python quant bindings with 2-bit QDQ helper for parity with the new path.
• Adds MLAS unit tests covering LUT GEMM across symmetric/asymmetric quant and multiple shapes/block sizes.

Main components:

• MlasInitLUTGemmKernelConfig: Config for LUT kernels

• MlasLUTGemmPackQuantBData: Pre Packing of quantized weight

• MlasLUTPackScalesAndZeroPoints: Pre Packing of qunatized scales and zero points

• MlasLUTGemm: Main Entry point

• GenerateLUT_avx2: LUT construction from activations

• TMACComputeGemm_avx2: AVX2 LUT GEMM kernel

• Session option: mlas.use_lut_gemm

How to test

• MLAS LUT GEMM unit tests: see test_sqlutgemm.cpp
• Run MatMulNBits models with session option mlas.use_lut_gemm=1 on AVX2 machines; expect fallback to existing path if availability checks fail.

Perf

Focus of this PR is functional + kernel bring-up; perf to be reported separately once broader profiling is done.

Future Work

• Support MLFloat16 (FP16 scales and zero points)
• Add neon kernel for ARM.
• Add kernels for 4 bit weights and bitnet kernels
• Broader batch (N>1) support and additional shape coverage.

[github-actions]

You can commit the suggested changes from lintrunner.

[edgchen1]

what is the overhead like for creating a new threadpool in each call to PrePack()?

I wonder if we should make an existing threadpool available to this code. perhaps we can pass in the threadpool from SessionState. something to consider, and maybe for a future PR.

[vraspar]

I agree, passing thread pool to PrePack would be clean. I am planning to create second PR improving Prepacking logic in general, I will include this along with this :)

[hariharans29]

IIUC - The usage of threadpool in the existing non-LUT path seems like a new addition - is that intentaional (and come with apprioriate tests) ?

[vraspar]

Initially, I thought tests in test_sqnbitgemm.cpp should suffice since they already test it with thread pool. I applied changes to only use thread pool for LUT path now.

Once we add tests, I think it might be beneficial to use thread pool for pre packing for other paths