The weird behavior of Batched Faster-Whisper

[toanhuynhnguyen]

I use this function "batched_model.transcribe":

from faster_whisper import BatchedInferencePipeline, WhisperModel

model = WhisperModel(
    model_size_or_path="large-v3",
    device="cuda",
    compute_type="float16",
)

batched_model = BatchedInferencePipeline(model=model)

segments, info = batched_model.transcribe(
    audio="file_a.wav" ,
    without_timestamps=True, # Change this value to True or False to test.
    word_timestamps=True,
    beam_size=1,
    task="transcribe",
    language="en",
    batch_size=5,
)

for segment in segments:
    print("[%.2fs -> %.2fs] %s" % (segment.start, segment.end, segment.text))

I have 2 audio files: file_a.wav and file_b.wav:

1. Transcribe the file_a.wav

• If I set without_timestamps=True for the "batched_model.transcribe" function then it transcribes all sentences in the audio file (My expectation).
• If I set without_timestamps=False then it misses a few sentences in the audio file.

2. Transcribe the file_b.wav:

• If I set without_timestamps=True then it misses a few sentences in the audio file.
• If I set without_timestamps=False then it transcribes all sentences in the audio file (My expectation).

The argument "without_timestamps" does not work consistently. How can I resolve this issue? Thank you so much.

[toanhuynhnguyen]

@MahmoudAshraf97 Can you help me with this issue, thank you so much.

[MahmoudAshraf97]

This is CT2 bug, it's not solvable in FW, you can reproduce the same bug with the non-batched inference

[toanhuynhnguyen]

Is there any solution or alternative for this issue?

[MahmoudAshraf97]

do not enable timestamps, if you need smaller segments, you can later split them using any other methods such as This

[toanhn]

You mean: set without_timestamps=True, right?

[MahmoudAshraf97]

Right

[toanhuynhnguyen]

But in the case (2) of transcribing the file_b.wav, I still get that issue:

Do you have any suggestions? Thanks, friend.