How to use VAD decoding in FunASR

[lingyunfly]

There are two decoding mode for VAD model(offline、 online)，for more details, please refer to VAD.

If you want to decode in offline mode

from modelscope.pipelines import pipeline
from modelscope.utils.constant import Tasks

if __name__ == '__main__':
    audio_in = 'https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/test_audio/vad_example.wav'
    output_dir = None
    inference_pipline = pipeline(
        task=Tasks.voice_activity_detection,
        model="damo/speech_fsmn_vad_zh-cn-16k-common-pytorch",
        model_revision='v1.2.0',
        output_dir=output_dir,
        batch_size=1,
    )
    segments_result = inference_pipline(audio_in=audio_in)
    print(segments_result)

If you want to decode in online mode, you should set the mode to online. The step means the package size send each time. The is_final means whether it is the last package of input data. When a new request is started, you should set the param_dict = {'in_cache': dict()}

from modelscope.pipelines import pipeline
from modelscope.utils.constant import Tasks
from modelscope.utils.logger import get_logger
import logging
logger = get_logger(log_level=logging.CRITICAL)
logger.setLevel(logging.CRITICAL)
import soundfile
import os


if __name__ == '__main__':
    os.environ["MODELSCOPE_CACHE"] = "./"
    output_dir = None
    inference_pipline = pipeline(
        task=Tasks.voice_activity_detection,
        model="damo/speech_fsmn_vad_zh-cn-16k-common-pytorch",
        model_revision='v1.2.0',
        output_dir=output_dir,
        batch_size=1,
        mode='online',
    )
    model_dir = os.path.join(os.environ["MODELSCOPE_CACHE"], "damo/speech_fsmn_vad_zh-cn-16k-common-pytorch")
    wav_file = os.path.join(model_dir, "example/vad_example.wav")
    speech, sample_rate = soundfile.read(wav_file)
    speech_length = speech.shape[0]
    
    sample_offset = 0
    step = 160 * 10
    param_dict = {'in_cache': dict()}
    for sample_offset in range(0, speech_length, min(step, speech_length - sample_offset)):
        if sample_offset + step >= speech_length - 1:
            step = speech_length - sample_offset
            is_final = True
        else:
            is_final = False
        param_dict['is_final'] = is_final
        segments_result = inference_pipline(audio_in=speech[sample_offset: sample_offset + step],
                                            param_dict=param_dict)
        print(segments_result)

[walkeronwater]

Hi, could you please give a short example to refresh timestamp cache that calling an online-mode VAD returns?

"When a new request is started, you should set the param_dict = {'in_cache': dict()}"
-- I've tried this method but got accumulated timestamps that never refreshed.