nbest C++ api use too much vram

[binhtranmcs]

I tried hlg decoding using both hlg_decode.cu and hlg_decode.py, modified them a bit to get nbest from the lattice. The python api seems fine but i got OOM when using c++ api. As i know, python api is still c++ under the hood. So I wonder if you all face the same issue as mine, that the c++ api use a lot more memory than python api, or I installed k2 incorrectly in some ways. Please help me with this.

Model: librispeech conformer ctc
Code:
python: hlg_decode.py.txt
c++: hlg_decode.cu.txt
Audio (change ext to .wav): testvram.txt

Thanks in advance!

[binhtranmcs]

Any updates on this?

[binhtranmcs]

@pkufool @csukuangfj do have any insight into this

[danpovey]

sorry for delay, if you could.find.a way to.display your diff conveniently it may be easier for.us.to look...

…

On Thursday, May 2, 2024, binhtranmcs ***@***.***> wrote: @pkufool <https://github.com/pkufool> @csukuangfj <https://github.com/csukuangfj> do have any insight into this — Reply to this email directly, view it on GitHub <#1281 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAZFLO3HIGRUI2PYIOBJ223ZAHUYHAVCNFSM6AAAAABGSPQDLSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOBZHAYTINRSGI> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[pkufool]

@binhtranmcs Have you figured out the reasons? I think it is because python breaks a large batch into smaller sub batches when doing intersect, see _intersect_device in the python code.

[binhtranmcs]

@binhtranmcs Have you figured out the reasons? I think it is because python breaks a large batch into smaller sub batches when doing intersect, see _intersect_device in the python code.

tks @pkufool, I will have a look at this