[EDGE CASE] Cartoon voice and voice-line musical instrument worse performance on v5 than older version

[George0828Zhang]

🐛 Bug

V5 ignores cartoon voices.

To Reproduce

Steps to reproduce the behavior:

1. Using colab example
2. Download this example and run until this cell (change 'en_example.wav' to 'ja_example.wav'):

wav = read_audio('ja_example.wav', sampling_rate=SAMPLING_RATE)
# get speech timestamps from full audio file
speech_timestamps = get_speech_timestamps(wav, model, sampling_rate=SAMPLING_RATE)
pprint(speech_timestamps)

3. The result is:

[{'end': 30464, 'start': 12032}]

while if old version is used (see SYSTRAN/faster-whisper#934 (comment)), the result is

[{'end': 40192, 'start': 12032},
 {'end': 179456, 'start': 76544},
 {'end': 379136, 'start': 273152},
 {'end': 457984, 'start': 422656},
 {'end': 630016, 'start': 576256},
 {'end': 669952, 'start': 653056},
 {'end': 863488, 'start': 695040},
 {'end': 950528, 'start': 896768}]

Expected behavior

V5 should be better than older version.

Environment

Please copy and paste the output from this
environment collection script
(or fill out the checklist below manually).

You can get the script and run it with:

wget https://raw.githubusercontent.com/pytorch/pytorch/master/torch/utils/collect_env.py
# For security purposes, please check the contents of collect_env.py before running it.
python collect_env.py

• PyTorch Version (e.g., 1.0):
• OS (e.g., Linux):
• How you installed PyTorch (conda, pip, source):
• Build command you used (if compiling from source):
• Python version:
• CUDA/cuDNN version:
• GPU models and configuration:
• Any other relevant information:

Additional context

[snakers4]

Can you send an audio sample?

[George0828Zhang]

Here:
https://drive.google.com/file/d/1NPvEybP0VU1dFmd6neH6JJRW_Qm2MXdk/view

thanks for looking into this!

[zh-plus]

Is there a workaround for this issue? I'm experiencing the same degradation in Japanese whispering audio.

[kuang-kuang]

Is there a workaround for this issue? I'm experiencing the same degradation in Japanese whispering audio.

I meet the same problem.

[snakers4]

Linked with #663

🐛 Bug

During the tests, it was revealed that the sounds of musical instruments of the string group, such as the piano and guitar, cause false alarms, considering the input signal to be a voice.

To Reproduce

Steps to reproduce the behavior:

1. Get a test audio example from here
2. Build and run an example silero-vad-onnx.cpp
3. Make sure that this result is returned:
   Speech detected from 0.2 s to 0.9 s Speech detected from 1.1 s to 4.8 s

Expected behavior

In this case, there should be such a result:
Speech detected from 3.5 s to 4.6 s