WSFT based decoding algorithm for Transducers with GPU

[manickavela29]

Hi @csukuangfj,

has there been any attempt to implement WFST based decoding algorithm with beam search or greedy?
It will be a great saver on latency, definitely you are the expert here, is there any blocker for it?

do you think for any reason, it will not be helpful in increasing the systems throughput or latency?

[csukuangfj]

has there been any attempt to implement WFST based decoding algorithm with beam search or greedy?

That is already supported in sherpa-onnx. You can use WFST graphs, e.g., HLG, with CTC models in sherpa-onnx.
Both streaming and non-streaming decoding are supported, though they work only on CPU in sherpa-onnx.

If you want to use WFSTs on GPU with CTC models, please see k2-fsa/sherpa. It has already been supported.

[manickavela29]

yes alright, great, went through them just now
how about Transducer models? there are greedy and modified beam search algos, is it possible to implement them with WFST.
contextual biasing is also part of beam search.

I am no expert with K2, but is it possible to have them with WFST or is there any limitation that I should know of

[manickavela29]

Hi @csukuangfj
just checking in again, sometimes there wouldn't be any notification with replies, so just taking and double checking

[csukuangfj]

you can have a look at our hotword doc. it is possible to implement it with WFST, I think.