Vad in Python

[thewh1teagle]

I'm trying to use silero vad with sherpa onnx for simple speech detection but it doesn't detect speech / end of speech correctly.

# pip install sherpa-onnx pvrecorder
# wget https://github.com/k2-fsa/sherpa-onnx/releases/download/asr-models/silero_vad.onnx
# python main.py

from pvrecorder import PvRecorder
import sherpa_onnx

# Vad
SAMPLE_RATE = 16000
config = sherpa_onnx.VadModelConfig()
config.silero_vad.model = 'silero_vad.onnx'
config.sample_rate = SAMPLE_RATE
vad = sherpa_onnx.VoiceActivityDetector(config, buffer_size_in_seconds=30)

# Recorder
buffered_frames_count = (SAMPLE_RATE * 10) // config.silero_vad.window_size # 10 seconds
recorder = PvRecorder(frame_length=config.silero_vad.window_size, buffered_frames_count=buffered_frames_count)
recorder.start()

is_speeching = False
while recorder.is_recording:
    frame = recorder.read()
    vad.accept_waveform(frame)
    # Speech detected
    if vad.is_speech_detected():
        if not is_speeching:
            is_speeching = True
            print('Speech')
    # End of speech
    elif is_speeching:
        is_speeching = False
        print('Done speech')
    # Clear vad buffer
    vad.flush()

Am I missing something?
Thanks

[csukuangfj]

please read our vad examples carefully.

the samples sent to accept_waveform must be exactly window_size 512

[thewh1teagle]

It's 512

PvRecorder(frame_length=config.silero_vad.window_size, buffered_frames_count=buffered_frames_count)

[csukuangfj]

in that case, please remove vad.flush()

you only need to call that at the very end.

[thewh1teagle]

Even when removing the flush it detects incorrectly speech and incorrectly silence. I double checked that the audio is valid by saving it to wav file.

I also tried with pymicro_vad and it worked with it.

[csukuangfj]

have you tried our c++ binary for vad?

[thewh1teagle]

No but I tried from Rust previously and I had the same issue as with Python

[csukuangfj]

https://github.com/k2-fsa/sherpa-onnx/blob/master/python-api-examples/vad-microphone.py

Have you tried this example?

[thewh1teagle]

The example works. I'll try to check what's different

[csukuangfj]

recorder = PvRecorder(frame_length=config.silero_vad.window_size, buffered_frames_count=buffered_frames_count)

Please never, never, use the default values of the microphone.

You have to at least specify

• number of channels, e.g., 1
• sample rate, e.g., 16000
• sample format, e.g., float32

If you don't know what are the default values, you have to at least try to find them out instead of blindly using them as they are.

[csukuangfj]

Without additional information, we are unfortunately not sure how to resolve this discussion. We are therefore reluctantly going to close this discussion for now. Please don't hesitate to comment on the discussion if you have any more information for us; we will reopen it right away!