Onnxrt execution provider configs

[manickavela29]

Config files handling for onnxrt provider
cc : #40

[manickavela29]

Hi,
I haven't tried with any Offlinemodel, so I am extending the config support to only for OnlineModel,
but easily it can be extended for OfflineModel.
Let me know if you have any better suggestions for maintaining the config.

session.cc files also has several other models (vad,etc), since these are minimal models, they are left as they are
if the support for trt/cuda backends can be extended for them.
let me know

for pybind interface, still adding in changes

[csukuangfj]

Hi, I haven't tried with any Offlinemodel, so I am extending the config support to only for OnlineModel, but easily it can be extended for OfflineModel. Let me know if you have any better suggestions for maintaining the config.

session.cc files also has several other models (vad,etc), since these are minimal models, they are left as they are if the support for trt/cuda backends can be extended for them. let me know

for pybind interface, still adding in changes

Thanks! That sounds good to me.

[csukuangfj]

Please have a look at errors reported by clang-tidy
https://github.com/k2-fsa/sherpa-onnx/actions/runs/9665507110/job/26662771827

and the style check by cpplint.

https://github.com/k2-fsa/sherpa-onnx/actions/runs/9665507136

(Note: You can run these two tools locally.)

[manickavela29]

I wanted to keep provider config unique for each models, this would give good control for CUDA and TRT configs,
But if developers give different devices(outside of cuda/trt) is a complex scenario and it must be handled, so I remove those patches midway.

Most of the fixes I had also went away with those patches thats why several bugs

[manickavela29]

Please have a look at errors reported by clang-tidy https://github.com/k2-fsa/sherpa-onnx/actions/runs/9665507110/job/26662771827

and the style check by cpplint.

https://github.com/k2-fsa/sherpa-onnx/actions/runs/9665507136

(Note: You can run these two tools locally.)

yes, have been using for cpplint, had a hard time with exit 🙂
will try with clangtiddy

[csukuangfj]

By the way, you can use

pip install clang-tidy

[manickavela29]

I ran the python tests in debug mode and got this error

• python3 ./python-api-examples/offline-telespeech-ctc-decode-files.py
  Traceback (most recent call last):
  File "/mnt/efs/manickavela/asr_work/trt/sherpa-onnx/./python-api-examples/offline-telespeech-ctc-decode-files.py", line 16, in
  import sherpa_onnx
  File "/mnt/efs/manickavela/miniconda3/envs/icefallmk/lib/python3.9/site-packages/sherpa_onnx/init.py", line 1, in
  from _sherpa_onnx import (
  ImportError: arg(): could not convert default argument 'trt_config: sherpa_onnx::TensorrtConfig' in method '<class '_sherpa_onnx.ProviderConfig'>.init' in
  to a Python object (type not registered yet?)

it would be great to have some hints for fixing this
I tried to take reference from vad /online model config implementation and attempted but it didn't help.
I will dump a patch with my experiments and later clear it

[manickavela29]

Hi @csukuangfj, would be great to have some leads on python fix

[csukuangfj]

please also change
https://github.com/k2-fsa/sherpa-onnx/blob/master/sherpa-onnx/python/sherpa_onnx/online_recognizer.py#L174

I give you an example below. You need to change other def from_xxx() accordingly.

Change

sherpa-onnx/sherpa-onnx/python/sherpa_onnx/online_recognizer.py

Lines 174 to 183 in 125bb9f

	model_config = OnlineModelConfig(
	transducer=transducer_config,
	tokens=tokens,
	num_threads=num_threads,
	provider=provider,
	model_type=model_type,
	modeling_unit=modeling_unit,
	bpe_vocab=bpe_vocab,
	debug=debug,
	)

to

        provider_config = ProviderConfig(provider=provider)

        model_config = OnlineModelConfig(
            transducer=transducer_config,
            tokens=tokens,
            num_threads=num_threads,
            provider_config=provider_config,
            model_type=model_type,
            modeling_unit=modeling_unit,
            bpe_vocab=bpe_vocab,
            debug=debug,
        )

Remember to change

sherpa-onnx/sherpa-onnx/python/sherpa_onnx/online_recognizer.py

Line 9 in 125bb9f

OnlineModelConfig,

 ProviderConfig,
 OnlineModelConfig, 

[manickavela29]

Hi @csukuangfj,
would be great if you can have one more review by today. 🙂

[csukuangfj]

Thank you!

Left some minor comments. Otherwise, it looks good to me.

[manickavela29]

Hi @csukuangfj,

I think it is all good now, expected tests are also passing through,
let me know if there are any more reviews

[csukuangfj]

Thank you for your contribution!

[manickavela29]

uffff, Thank you so much