[WIP] Export CTC decoding algorithm to sherpa

[pkufool]

See k2-fsa/sherpa#177 for our plans.

• Move https://github.com/k2-fsa/k2/blob/master/k2/csrc/torch_api.h to https://github.com/k2-fsa/k2/tree/master/k2/torch/csrc
• Remove the dependency kaldifeat_core from k2_torch. We can make the binaries depend on kaldifeat_core directly.
• Export the library k2_torch in

  k2/cmake/k2Config.cmake.in

  Line 48 in c3a7404

  set(K2_LIBRARIES k2_torch_api k2_log k2context k2fsa)

• Add the following functions to torch_api.h
  • A function to load HLG.pt. The returned type can be std::shared_ptr<k2::FsaClass>. We can define an alias FsaClassPtr for it, like RaggedShapePtr
  • A function to wrap k2::CtcTopo(). It should also return a value of type FsaClassPtr.
  • A function for CTC/HLG decoding. It takes the following inputs:
    - log_softmax_out: a 3-D tensor of shape (N, T, C)
    - log_softmax_out_lens, a 1-D tensor of shape (N,)
    - FsaClassPtr, can be either a CtcTopo or an HLG
    and it returns std::vector<std::vector<int32_t>>

[csukuangfj]

Suggested change

DESTINATION

[csukuangfj]

Suggested change

	install(TARGETS k2_torch_api
	install(TARGETS k2_torch_api k2_torch

[csukuangfj]

It will recompile feature_srcs for each binary. Shall we make it a library that can be shared?

[pkufool]

OK, I thinks so. so, let's change back to its original

[csukuangfj]

I suggest that we create a new PR to bind the exposed APIs to Python and provide python APIs and examples to decode models trained using CTC loss from various frameworks, such as icefall, nemo, espnet, speechbrain, wenet, etc.

[pkufool]

I suggest that we create a new PR to bind the exposed APIs to Python and provide python APIs and examples to decode models trained using CTC loss from various frameworks, such as icefall, nemo, espnet, speechbrain, wenet, etc.

I think we MUST create a new PR, because the binding and demo code will be in sherpa, not k2.

[csukuangfj]

k2 only requires log_softmax_out + TLG for decoding, so it is easier to use.
People can just import k2 and provide the required inputs for decoding and they may not want to install sherpa.

[pkufool]

k2 only requires log_softmax_out + TLG for decoding, so it is easier to use. People can just import k2 and provide the required inputs for decoding and they may not want to install sherpa.

OK, I got your idea, I think to do that the functions to be wrapped is in k2/torch/* not in torch_api.h. It does not relate to this PR, will do it in another change, thanks!

[csukuangfj]

I suggest that we provide only a single method Decode() to give the results directly.

In the current approach, users have to call BestPath on the returned lattice and that is the only function that users can call for the returned lattice.

That is an implementation detail and we can hide it from the users.

[pkufool]

I suggest that we provide only a single method Decode() to give the results directly.

In the current approach, users have to call BestPath on the returned lattice and that is the only function that users can call for the returned lattice.

That is an implementation detail and we can hide it from the users.

I thought we can use this lattice to do rescoring, we can implement one_best_decoding and ngram_rescoring in sherpa.

[pkufool]

I think there is no need to hide lattice.