question about disable_per_channel_for_dense_layers in convert.py

[TeahyungKim]

Hello, I have a question about disable_per_channel_for_dense_layers.

I’m currently using QuantizationDebugger for quantization, and in this part of the code, the convert.mlir_quantize() function is called with the disable_per_channel_for_dense_layers argument set to its default value.
As a result, it is always disable_per_channel_for_dense_layers=False, which is causing an issue.

The real problem is that it’s set to this default value. Although the option is deprecated, it still causes problems for users like me who are using NNAPI. This eventually leads to the following error in android.

08-28 12:03:58.063 E 23797 23797 OperationValidationUtils: NN_RET_CHECK failed (packages/modules/NeuralNetworks/common/types/src/OperationsValidationUtils.cpp:38type == expectedTypes[i] Invalid input tensor type TENSOR_QUANT8_SYMM_PER_CHANNEL for input 1, expected TENSOR_QUANT8_ASYMM_SIGNED 08-28 12:03:58.063 E 23797 23797 ModelBuilder: Invalid Operation: NN_RET_CHECK failed (packages/modules/NeuralNetworks/common/types/operations/src/FullyConnected.cpp:120 validateInputTypes(context, inExpectedTypes)

I’d like to ask if it would be possible to set this option’s default back to True, as it was before tf 2.17.

[gaikwadrahul8]

Hi, @TeahyungKim
The recommended solution is to explicitly disable per-channel quantization for dense layers to ensure maximum compatibility. You can achieve this by setting disable_per_channel_for_dense_layers=True during the quantization call. Since you are using the QuantizationDebugger you'll need to generate the debug model with this flag enabled before passing it to the debugger something like below

# After creating the calibrated_model from the converter
debug_model_content = convert.mlir_quantize(
    calibrated_model,
    disable_per_channel_for_dense_layers=True,
    enable_numeric_verify=True  # Ensure debug ops are included
)

# Initialize the debugger with the already quantized model content
quant_debugger = debugger.QuantizationDebugger(
    quant_debug_model_content=debug_model_content,
    debug_dataset=representative_dataset_gen
)
quant_debugger.run()

While we understand the request to revert the default the current setting was a deliberate choice to prioritize accuracy on newer hardware. Providing this explicit flag allows developers to balance accuracy and broad hardware compatibility as needed. Please refer updated documentation of LiteRT Delegates and NNAPI Migration Guide. Thank you for your cooperation and understanding.