Skip to content

feat(supertonic): pick a latent-length bucket per piece - #144

Merged
ivan-digital merged 2 commits into
mainfrom
feat/supertonic-latent-buckets
Aug 30, 2026
Merged

feat(supertonic): pick a latent-length bucket per piece#144
ivan-digital merged 2 commits into
mainfrom
feat/supertonic-latent-buckets

Conversation

@ivan-digital

Copy link
Copy Markdown
Member

Summary

Follow-up to #141. The published LiteRT graphs are fixed at L=64 latent frames (~4.5 s), so a sentence the window cannot hold is split, and the second piece — a separate generation — restarts with sentence-initial prosody. That residual roughness cannot be fixed on the host; it needs a longer window.

soniqo/Supertonic-3-LiteRT now ships an L=128 bucket (vector_estimator_L128.tflite + vocoder_L128.tflite, ≈9 s; exported by speech-models' export_litert.py --latent-frames 64 128). This PR teaches LiteRTSupertonicTts to use it.

What changed

  • Latent buckets. *_L{N}.tflite siblings of the base graphs are discovered at construction (latent_buckets()) and loaded on first use, so memory only grows when a long piece needs one. Each piece runs on the smallest bucket whose window holds its predicted duration (choose_latent_bucket(), pure + unit-tested); the planner's window is the largest bucket. A bucket that fails to load is skipped with a log line and the piece falls back to a smaller window.
  • Inputs bound by tensor name. The converter permutes a graph's input slots away from the signature order, and differently per toolchain version: the published base graphs bind [text_mask, text_ids, style_dp], a current litert-torch export binds [text_ids, style_dp, text_mask]. Slots are now resolved from serving_default_args_N at load for all three multi-input graphs, with the published order as fallback. Without this, any re-export of the bundle would have broken at run time ("Failed to register input tensor buffer").
  • Docs: LiteRTSupertonicTts section in docs/models.md.

Test plan

Depends on the L=128 files being published to soniqo/Supertonic-3-LiteRT; without them behaviour is unchanged.

@ivan-digital
ivan-digital changed the base branch from fix/supertonic-chunking to main August 30, 2026 10:35
The published LiteRT graphs are fixed at L=64 (~4.5 s), so a sentence the
window cannot hold is split and the second piece restarts with
sentence-initial prosody. A bundle may now ship extra fixed-L exports of the
two L-dependent graphs next to the base ones (`vector_estimator_L128.tflite`
+ `vocoder_L128.tflite`, from speech-models' export_litert.py
--latent-frames 64 128). They are discovered at construction and loaded on
first use; each piece runs on the smallest bucket whose window holds its
predicted duration, so short pieces keep the cheap base graph and a long
sentence is generated in one pass. The planner's window becomes the largest
bucket, and a split only remains for text longer than that. A bucket that
fails to load is skipped with a log line. Without extra graphs the output is
unchanged (verified byte-identical).
The converter permutes a graph's input slots away from the signature order,
and differently per toolchain version: the published base graphs bind
[text_mask, text_ids, style_dp] where a current litert-torch export binds
[text_ids, style_dp, text_mask]. The hard-coded slot order therefore broke
any re-exported graph at run time ("Failed to register input tensor buffer"),
including the L=128 bucket. Resolve the slot per role from the tensor names
(`serving_default_args_N`) when a model is loaded, for the duration
predictor, the text encoder and every vector_estimator bucket, and keep the
published order as the fallback. Output for the published bundle is
unchanged (verified byte-identical).
@ivan-digital
ivan-digital force-pushed the feat/supertonic-latent-buckets branch from d41e996 to 01b8d51 Compare August 30, 2026 10:36
@ivan-digital
ivan-digital merged commit e5e4adc into main Aug 30, 2026
15 checks passed
@ivan-digital
ivan-digital deleted the feat/supertonic-latent-buckets branch August 30, 2026 10:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant