Skip to content

Punctuation tokens book real duration and fill it with audible signal (voiced junk after final punctuation; next-word onsets inside leading quote tokens) #365

Description

@AzaleaHarper

Summary: Kokoro books real duration for punctuation tokens and fills those frames with audible signal — voiced junk after sentence-final punctuation, and (the reverse) the next word's onset realized early inside a leading quote-glyph token. Both are invisible in casual use but show up constantly in long-form/audiobook generation, and both are precisely measurable with the per-token durations from KModel.forward_with_tokens / the ONNX timestamped path.

Setup: Kokoro-82M (v1.0 weights), espeak/misaki en-gb phonemes, per-sentence generation. We sum the per-token duration frames to get each phoneme token's exact audio span, so signal can be attributed to the token whose booked frames contain it.

Finding 1 — trailing punctuation carries vocalization. For dialogue like "The paw!" the ! and closing " tokens together booked ~677 ms, and the audio inside those frames is not silence or release tail: we measured sustained voiced signal at about −19 dBFS, with mid-span energy re-rises well after the final phone ended. Perceptually it's a breathy/static tail glued to every exclamation or quoted-dialogue boundary. On a TV speaker at audiobook listening levels it is clearly audible, and in a full book it recurs at nearly every dialogue boundary (." and !" endings).

Finding 2 — leading quote tokens carry the next word's onset. For a sentence starting "I threw it on the fire. the opening " token booked 57 ms, and the rising vowel onset of I begins inside the pad + " frames — periodicity of that region measures 0.84 (clearly voiced onset, not noise). So the attack of the first word starts ahead of the word's own booked frames. Anything that trims or gates "non-speech" tokens by the duration clock will amputate word onsets (we learned this the hard way: gating the quote token's frames produced an audible click and a 30–40 ms hole at every quoted line).

Why it matters: the duration clock is otherwise excellent — accurate enough that we build word-level read-along timing from it — but for punctuation tokens the booked frames and the semantic content disagree in both directions. Anyone doing token-aligned post-processing (silence gating, subtitle timing, splice points) will hit both cases.

Workaround we settled on: treat trailing punctuation spans with an energy-content test (keep a ~100 ms release grace, gate only spans whose content re-rises or sustains above −45 dBFS), and never gate the tail of a punctuation span that is followed by speech — walk back from the span end while energy stays above floor and treat that run as the next word's attack.

Happy to provide exact clips/measurements if useful.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions