[RNE Rewrite] feat: add supertonic TTS pipeline by barhanc · Pull Request #1317 · software-mansion/react-native-executorch

barhanc · 2026-07-17T14:28:07Z

Description

Add supertonic TTS pipeline and example app screen to test it.

Introduces a breaking change?

Yes
No

Type of change

Bug fix (change which fixes an issue)
New feature (change which adds functionality)
Documentation update (improves or adds clarity to existing documentation)
Other (chores, tests, code style improvements etc.)

Tested on

iOS
Android

Testing instructions

Run the Speech example app on both iOS and Android and test the newly added TTS functionality.

Screenshots

Related issues

Part of #1250

Checklist

I have performed a self-review of my code
I have commented my code, particularly in hard-to-understand areas
I have updated the documentation accordingly
My changes generate no new warnings

Additional notes

Should wait for #1328 so that the hook implementation can be refactored to a more concise and elegant one.

msluszniak · 2026-08-06T06:53:23Z

+   */
+  synthesize: (
+    text: string,
+    options: SupertonicTtsOptions<K>


What is our convention right now, when do we use opts and when options?

In all task pipeline implementations we use options right now for user-facing API. In helpers we mostly use opts but there are some inconsistencies there. I think we should make them all use one or the other (which one would you prefer?), but this can be done in a follow-up style PR.

I'm in favour of options everywhere. Opts suffixes in config object field and types are fine I guess. Also when we add PR with refactor we should add note in core-guidelines/SKILL.md.

msluszniak

Main thing is partition(), I found three cases where it misbehaves (comments inline). The first one fires on pretty much any input that doesn't end in punctuation, so I'd want that sorted before merge.

msluszniak · 2026-08-06T12:26:48Z

When I set the slowest speed aka 0.7 I got in half of the cases strange artefacts with repetitions of some parts of sentences. Exact parameters for reproduction:
device: Android
backend: Xnnpack
voice: M1
language: pl
speed: 0.7
denoising steps: 8

Sentence: "Cześć co tam słychać? Testuję funkcje mowy w naszej demo aplikacji."

EDIT: When I tested the same configuration but with speed 2.0, the results were even worse :((

barhanc · 2026-08-06T12:37:51Z

When I set the slowest speed aka 0.7 I got in half of the cases strange artefacts with repetitions of some parts of sentences. Exact parameters for reproduction: device: Android backend: Xnnpack voice: M1 language: pl speed: 0.7 denoising steps: 8

Sentence: "Cześć co tam słychać? Testuję funkcje mowy w naszej demo aplikacji."

EDIT: When I tested the same configuration but with speed 2.0, the results were even worse :((

Yeah, from my experiments the extreme values for speed are often hit-or-miss depending on the voice and text to synthesize. I checked the Transformers.js implementation of Supertonic (https://huggingface.co/spaces/webml-community/Supertonic-TTS-WebGPU) and they only allow speeds in [0.8; 1.2]. This is a limitation of the model I believe so we can narrow the available speed range.

msluszniak · 2026-08-06T12:39:00Z

Yeah, I'm in favour of limiting speed min and max.

msluszniak

LGTM! If this PR is fine for you @IgorSwat ,we just need to rebase it with resource fetcher PR and ship :))

IgorSwat

Text partitioner in this form is unacceptable. It's not prioritizing low latency (time to first audio chunk) as the old one does. And it will hurt the performance (particularly for Kokoro) very seriously.

I would just add a mechanic (like a flag or smth) that if set, makes the algorithm prioritize splitting heavily at the beginning and slowly disable it as we process more and more of text.

msluszniak · 2026-08-06T15:22:33Z

For Kokoro, there will be a separate flow I guess, since these models are too specific to support them both under same functionality.

IgorSwat · 2026-08-06T15:25:10Z

For Kokoro, there will be a separate flow I guess, since these models are too specific to support them both under same functionality.

In current codebase same TextPartitioner is used for both. And works fine.

…rettify DP comment

…up AudioContext

…ate in Supertonic TTS

… text preprocessing

…tput

…s in textPartitioner

… text partitioner

barhanc · 2026-08-06T16:27:15Z

Text partitioner in this form is unacceptable. It's not prioritizing low latency (time to first audio chunk) as the old one does. And it will hurt the performance (particularly for Kokoro) very seriously.

I would just add a mechanic (like a flag or smth) that if set, makes the algorithm prioritize splitting heavily at the beginning and slowly disable it as we process more and more of text.

I've added the configurable mechanism to prioritize splitting at the beginning and also made other options of the partitioner configurable.

msluszniak · 2026-08-06T16:30:12Z

@barhanc Would we share partitioner part across supertonic and kokoro?

barhanc · 2026-08-06T16:31:44Z

Yes, right now it is very similar to the main branch one, and configurable options make it adjustable for different models.

msluszniak · 2026-08-06T16:51:31Z

Thank you @IgorSwat for this on point observation. Please check if now everything is fine. Btw, maybe you want to add Kokoro to the new flow as a flow-up? ;)

IgorSwat · 2026-08-06T18:57:40Z

Thank you @IgorSwat for this on point observation. Please check if now everything is fine. Btw, maybe you want to add Kokoro to the new flow as a flow-up? ;)

I will adjust my Kokoro PR after this one is merged.

Rebasing onto the supertonic TTS pipeline (#1317) surfaced two gaps the suites themselves reported: - The registry walk assumed every task config names a single `modelPath`. Supertonic assembles four `.pte` files under `modelPaths`, so its category looked empty and its variant group went unchecked. Recognize both shapes, and compare variants on the whole set of files they name. - The export snapshots record the new TTS surface. Additions only — no removal or rename.

Addresses the review threads on #1354. Codes are no longer generated (barhanc): `scripts/errors.config.ts` and `generate-errors.ts` are deleted along with the codegen script and CI drift check. `src/core/error.ts` is now the source of truth and `cpp/core/error.h` mirrors it by hand, like every other part of the TS/JSI interface. TypeScript: - Codes are a string union, not a numeric enum, and the set shrinks from 13 to 10. A code now has to justify a distinct recovery path; the tokenizer and not-supported codes folded into the general categories. - One `RnExecuTorchError(code, message)` factory replaces the class/worklet-helper pair. It is a function, so it works unchanged on both runtimes rather than needing two spellings. - `isRnExecuTorchError(err, code?)` takes an optional code to narrow. - The internal fetcher `AbortError` class becomes a DOWNLOAD_ABORTED error; `useResourceDownload` matches the code instead of the class. C++: - `CodedError` -> `RnExecuTorchException`, `ErrorCode` -> `RnExecuTorchErrorCode`, `etCode` -> `etRuntimeErrorCode`. - `throwJs` and `makeJsError` are replaced by a single `throwJsiRnExecuTorchError(rt, e)` that only takes the C++ exception, so there is no way to reach JS without constructing one first. - `unwrapEt` leaves the error namespace and goes back to being a file-local `unwrap` helper with a single signature, no jsi::Runtime. Example apps are reverted to their original state: they are a testing ground and should surface raw errors (barhanc, #1288). The new Supertonic TTS throw sites from #1317 are converted to the convention. Agent skills are updated to match.

## Description Changes naming from `opts` to `options` across the board in TS API. Follow-up to [comment](https://github.com/software-mansion/react-native-executorch/pull/1317#:~:text=I%27m%20in%20favour%20of%20options%20everywhere.%20Opts%20suffixes%20in%20config%20object%20field%20and%20types%20are%20fine%20I%20guess.%20Also%20when%20we%20add%20PR%20with%20refactor%20we%20should%20add%20note%20in%20core%2Dguidelines/SKILL.md.) from #1317 ### Introduces a breaking change? - [ ] Yes - [x] No ### Type of change - [ ] Bug fix (change which fixes an issue) - [ ] New feature (change which adds functionality) - [ ] Documentation update (improves or adds clarity to existing documentation) - [x] Other (chores, tests, code style improvements etc.) ### Tested on - [ ] iOS - [ ] Android ### Testing instructions N/A ### Screenshots  ### Related issues  ### Checklist - [ ] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have updated the documentation accordingly - [ ] My changes generate no new warnings ### Additional notes

Addresses the review threads on #1354. Codes are no longer generated (barhanc): `scripts/errors.config.ts` and `generate-errors.ts` are deleted along with the codegen script and CI drift check. `src/core/error.ts` is now the source of truth and `cpp/core/error.h` mirrors it by hand, like every other part of the TS/JSI interface. TypeScript: - Codes are a string union, not a numeric enum, and the set shrinks from 13 to 10. A code now has to justify a distinct recovery path; the tokenizer and not-supported codes folded into the general categories. - One `RnExecuTorchError(code, message)` factory replaces the class/worklet-helper pair. It is a function, so it works unchanged on both runtimes rather than needing two spellings. - `isRnExecuTorchError(err, code?)` takes an optional code to narrow. - The internal fetcher `AbortError` class becomes a DOWNLOAD_ABORTED error; `useResourceDownload` matches the code instead of the class. C++: - `CodedError` -> `RnExecuTorchException`, `ErrorCode` -> `RnExecuTorchErrorCode`, `etCode` -> `etRuntimeErrorCode`. - `throwJs` and `makeJsError` are replaced by a single `throwJsiRnExecuTorchError(rt, e)` that only takes the C++ exception, so there is no way to reach JS without constructing one first. - `unwrapEt` leaves the error namespace and goes back to being a file-local `unwrap` helper with a single signature, no jsi::Runtime. Example apps are reverted to their original state: they are a testing ground and should surface raw errors (barhanc, #1288). The new Supertonic TTS throw sites from #1317 are converted to the convention. Agent skills are updated to match.

barhanc self-assigned this Jul 17, 2026

barhanc added refactoring feature PRs that implement a new feature labels Jul 17, 2026

barhanc changed the title ~~feat: add supertonic helpers~~ [RNE Rewrite] feat: add supertonic TTS pipeline Jul 17, 2026

barhanc linked an issue Jul 17, 2026 that may be closed by this pull request

[RNE Rewrite] Speech - add Kokoro/Supertonic TTS pipeline implementation #1250

Open

barhanc force-pushed the @bh/tts branch from 1201a19 to a32d281 Compare July 20, 2026 13:01

barhanc force-pushed the @bh/tts branch from a32d281 to e519596 Compare August 4, 2026 10:21

barhanc marked this pull request as ready for review August 5, 2026 16:53

barhanc requested review from IgorSwat and msluszniak August 5, 2026 16:58

msluszniak reviewed Aug 6, 2026

View reviewed changes

Comment thread packages/react-native-executorch/src/extensions/speech/tasks/supertonicTextToSpeech.ts Outdated

Comment thread packages/react-native-executorch/src/extensions/speech/tasks/supertonicTextToSpeech.ts

msluszniak reviewed Aug 6, 2026

View reviewed changes

Comment thread packages/react-native-executorch/src/extensions/speech/tasks/supertonicTextToSpeech.ts

msluszniak reviewed Aug 6, 2026

View reviewed changes

Comment thread apps/speech/app/text-to-speech/index.tsx

msluszniak approved these changes Aug 6, 2026

View reviewed changes

IgorSwat requested changes Aug 6, 2026

View reviewed changes

barhanc added 8 commits August 6, 2026 18:08

feat: add supertonic helpers

e5061a0

Remove redundant gaussian noise generator, fix cspell warnings, and p…

52d420a

…rettify DP comment

Enforce limit in all partition() code paths

96e6340

feat(speech): implement Supertonic text-to-speech pipeline and hook

2d15b80

feat(apps/speech): add Supertonic TTS example screen

b3c1079

fix: update comment

e297e6a

feat: update model paths

7108028

fix(speech): resolve audio playback failure on model change and clean…

6770e6d

…up AudioContext

barhanc added 12 commits August 6, 2026 18:08

fix cspell warnings, rename useSupertonicTextToSpeech to useTextToSpeech

3442b89

fix: add japanese custom chunk length rule

75eaa4f

refactor(speech): replace createSynchronizable with isSynthesizing st…

9c9034d

…ate in Supertonic TTS

fix(speech): add validation guard for totalSteps in Supertonic TTS

e548e73

fix(speech): add terminal breakpoint in textPartitioner and two-phase…

53eb2d0

… text preprocessing

fix(speech): use exact PCM buffer duration in Supertonic TTS chunk ou…

6043971

…tput

add TODOs

db5c8ac

fix: disable MLX model option on Android in TTS screen

349cd80

narrow TTS speed range to [0.8, 1.2]

2253a04

fix(speech): align breakpoint index calculation with UTF-16 code unit…

08ea026

…s in textPartitioner

feat(speech): add configurable TTFA initial segment prioritization to…

6d23846

… text partitioner

fix(speech): update TTS pipeline to use react-native-blob-util

690037f

barhanc force-pushed the @bh/tts branch from 50c3ba9 to 690037f Compare August 6, 2026 16:24

barhanc requested a review from IgorSwat August 6, 2026 16:27

fix docs

0d01392

IgorSwat approved these changes Aug 7, 2026

View reviewed changes

barhanc merged commit 94521d9 into rne-rewrite Aug 7, 2026
0 of 2 checks passed

barhanc deleted the @bh/tts branch August 7, 2026 11:18

barhanc mentioned this pull request Aug 10, 2026

[RNE Rewrite] style(ts): rename opts to options #1358

Merged

12 tasks

Uh oh!

Conversation

barhanc commented Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Introduces a breaking change?

Type of change

Tested on

Testing instructions

Screenshots

Related issues

Checklist

Additional notes

Uh oh!

Uh oh!

msluszniak Aug 6, 2026

Choose a reason for hiding this comment

Uh oh!

barhanc Aug 6, 2026

Choose a reason for hiding this comment

Uh oh!

msluszniak Aug 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

msluszniak left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

msluszniak commented Aug 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

barhanc commented Aug 6, 2026

Uh oh!

msluszniak commented Aug 6, 2026

Uh oh!

msluszniak left a comment

Choose a reason for hiding this comment

Uh oh!

IgorSwat left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

msluszniak commented Aug 6, 2026

Uh oh!

IgorSwat commented Aug 6, 2026

Uh oh!

barhanc commented Aug 6, 2026

Uh oh!

msluszniak commented Aug 6, 2026

Uh oh!

barhanc commented Aug 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

msluszniak commented Aug 6, 2026

Uh oh!

IgorSwat commented Aug 6, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

barhanc commented Jul 17, 2026 •

edited

Loading

msluszniak Aug 6, 2026 •

edited

Loading

msluszniak left a comment •

edited

Loading

msluszniak commented Aug 6, 2026 •

edited

Loading

IgorSwat left a comment •

edited

Loading

barhanc commented Aug 6, 2026 •

edited

Loading