Adaptation using K2

[AlexandderGorodetski]

Hello guys,

I have 4500 hours of data transcribed with Whisper.

In addition I have 250 hours of data transcribed manually.

Can I perform initial epochs with 4500 hours and afterwards I can perform additional 10 epochs with 250 hours.

What do you think?

Thanks,
AlexG.

[sangeet2020]

Hey,

Training initial epochs 10 or 15 eochs with the weakly supervised data (such as the whisper transcripts) should work in principle. But it totally depends what you want to achieve - if you are targeting strong performance only on the manually transcribed data- go for fine-tuning. If your goal is to just complement your weakly supervised data with manually transcribed data to improve the overall performance, then using mux might be a good idea.

Could you also post your conclusions from results. Would be interesting to read.
thanks