Skip to content

fix(asr): end the final VAD speech segment at the last frame's end - #16200

Open
Kayvan-Zahiri wants to merge 2 commits into
NVIDIA-NeMo:mainfrom
Kayvan-Zahiri:fix/vad-binarization-final-frame
Open

fix(asr): end the final VAD speech segment at the last frame's end#16200
Kayvan-Zahiri wants to merge 2 commits into
NVIDIA-NeMo:mainfrom
Kayvan-Zahiri:fix/vad-binarization-final-frame

Conversation

@Kayvan-Zahiri

Copy link
Copy Markdown

What does this PR do ?

binarization() closes a trailing speech segment at i * frame_length_in_sec, where i is the index of the last active frame, so the segment ends where that frame starts. Interior segments close at the index of the first inactive frame, i.e. at the end of the last active frame. A segment still active at the end of the sequence therefore comes out one full frame short, and a lone active final frame yields a zero-duration segment [t, t].

binarization_vectorized in the same module does not have this. Its test final-active-frame-uses-full-duration expects [[0.0, 0.16]] for [0.6, 0.6] at 0.08 s frames; binarization returns [[0.0, 0.08]]. The affected callers are generate_vad_segment_table_per_tensor (frame-VAD speech segments and RTTM) and ts_vad_post_processing.

Collection: ASR

Changelog

  • binarization() now ends the trailing segment at len(sequence) * frame_length_in_sec + pad_offset.
  • Dropped the pre-loop i = 0, which only existed to feed that expression.
  • Added test_binarization_final_active_frame_uses_full_duration, which also asserts agreement with binarization_vectorized.

Usage

import torch
from nemo.collections.asr.parts.utils.vad_utils import binarization, binarization_vectorized

per_args = {'onset': 0.5, 'offset': 0.5, 'pad_onset': 0.0, 'pad_offset': 0.0, 'frame_length_in_sec': 0.08}

# Two speech runs of two frames each. Only the trailing one is truncated.
seq = torch.tensor([0.9, 0.9, 0.1, 0.9, 0.9])
binarization(seq, per_args)             # before: [[0.00, 0.16], [0.24, 0.32]]
binarization_vectorized(seq, per_args)  #         [[0.00, 0.16], [0.24, 0.40]]

# A single active final frame collapses to zero duration.
binarization(torch.tensor([0.1, 0.1, 0.9]), per_args)  # before: [[0.16, 0.16]]

Checks run

  • pytest tests/collections/speaker_tasks/utils/ tests/collections/asr/utils/ -m "not pleasefixme" --cpu -> 834 passed, 40 skipped. The three new cases fail before the fix and pass after.
  • A differential fuzz of 5000 random sequences against binarization_vectorized (onset >= offset, frame lengths 0.01/0.02/0.08 s) goes from 3268 mismatches and 851 zero-duration segments to 0 and 0. Empty results are skipped there because binarization returns shape [0] while binarization_vectorized returns (0, 2); that shape difference is left alone.
  • black --check, isort --check-only, flake8 --config=.flake8.speech, pylint --rcfile=.pylintrc.speech -> clean, pylint 10.00/10.
  • Skipped: GPU tests and the full ASR suite, neither reachable on this machine. binarization is @torch.jit.script and still compiles.

Before your PR is "Ready for review"

Pre checks:

  • Make sure you read and followed Contributor guidelines
  • Did you write any new necessary tests?
  • Did you add or update any necessary documentation?
  • Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)

PR Type:

  • New Feature
  • Bugfix
  • Documentation

Who can review?

@nithinraok per CONTRIBUTING.md for ASR.

binarization() closed a trailing speech segment at the start time of the
last active frame instead of its end, so any segment still active at the
end of the sequence came out one frame short, and a lone active final
frame produced a zero-duration segment.

Signed-off-by: Kayvan Zahiri <123409429+Kayvan-Zahiri@users.noreply.github.com>
@copy-pr-bot

copy-pr-bot Bot commented Sep 2, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@nithinraok nithinraok left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks LGTM. Verified locally.

@nithinraok

Copy link
Copy Markdown
Member

/ok to test 6cab274

@nithinraok
nithinraok enabled auto-merge (squash) September 2, 2026 15:49
@svcnvidia-nemo-ci svcnvidia-nemo-ci added waiting-on-customer Waiting on the original author to respond waiting-on-maintainers Waiting on maintainers to respond and removed waiting-on-customer Waiting on the original author to respond labels Sep 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ASR community-request waiting-on-maintainers Waiting on maintainers to respond

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants