Skip to content

feat: Kokoro 82M TTS engine + voice profile type system #325

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jamiepine merged 4 commits into from
Mar 20, 2026
Merged

feat: Kokoro 82M TTS engine + voice profile type system #325

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
3584283
feat: Kokoro 82M TTS engine + voice profile type system
jamiepine Mar 19, 2026
9e726ad
fix: remove engine dropdown filtering — profile grid handles it
jamiepine Mar 19, 2026
0fc2192
fix: resolve relative paths using configured data dir, not CWD
jamiepine Mar 19, 2026
d6f48ac
Mirror the regular /generate endpoint behavior more closely
jamiepine Mar 19, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 61 additions & 9 deletions app/src/components/Generation/EngineModelSelector.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ import {
SelectTrigger,
SelectValue,
} from '@/components/ui/select';
import type { VoiceProfileResponse } from '@/lib/api/types';
import { getLanguageOptionsForEngine } from '@/lib/constants/languages';
import type { GenerationFormValues } from '@/lib/hooks/useGenerationForm';

Expand All @@ -15,13 +16,14 @@ import type { GenerationFormValues } from '@/lib/hooks/useGenerationForm';
* Adding a new engine means adding one entry here.
*/
const ENGINE_OPTIONS = [
{ value: 'qwen:1.7B', label: 'Qwen3-TTS 1.7B' },
{ value: 'qwen:0.6B', label: 'Qwen3-TTS 0.6B' },
{ value: 'luxtts', label: 'LuxTTS' },
{ value: 'chatterbox', label: 'Chatterbox' },
{ value: 'chatterbox_turbo', label: 'Chatterbox Turbo' },
{ value: 'tada:1B', label: 'TADA 1B' },
{ value: 'tada:3B', label: 'TADA 3B Multilingual' },
{ value: 'qwen:1.7B', label: 'Qwen3-TTS 1.7B', engine: 'qwen' },
{ value: 'qwen:0.6B', label: 'Qwen3-TTS 0.6B', engine: 'qwen' },
{ value: 'luxtts', label: 'LuxTTS', engine: 'luxtts' },
{ value: 'chatterbox', label: 'Chatterbox', engine: 'chatterbox' },
{ value: 'chatterbox_turbo', label: 'Chatterbox Turbo', engine: 'chatterbox_turbo' },
{ value: 'tada:1B', label: 'TADA 1B', engine: 'tada' },
{ value: 'tada:3B', label: 'TADA 3B Multilingual', engine: 'tada' },
{ value: 'kokoro', label: 'Kokoro 82M', engine: 'kokoro' },
] as const;

const ENGINE_DESCRIPTIONS: Record<string, string> = {
Expand All @@ -30,11 +32,38 @@ const ENGINE_DESCRIPTIONS: Record<string, string> = {
chatterbox: '23 languages, incl. Hebrew',
chatterbox_turbo: 'English, [laugh] [cough] tags',
tada: 'HumeAI, 700s+ coherent audio',
kokoro: '82M params, CPU realtime, 8 langs',
};

/** Engines that only support English and should force language to 'en' on select. */
const ENGLISH_ONLY_ENGINES = new Set(['luxtts', 'chatterbox_turbo']);

/** Engines that support cloned (reference audio) profiles. */
const CLONING_ENGINES = new Set(['qwen', 'luxtts', 'chatterbox', 'chatterbox_turbo', 'tada']);

/** Engines that are preset-only (no cloning). */
const PRESET_ONLY_ENGINES = new Set(['kokoro']);

/**
* Get which engine options are available for the selected profile.
*
* - Preset profiles: locked to their preset engine
* - All other profiles: all engines available
*/
function getAvailableOptions(selectedProfile?: VoiceProfileResponse | null) {
if (!selectedProfile) return ENGINE_OPTIONS;

const voiceType = selectedProfile.voice_type || 'cloned';

if (voiceType === 'preset') {
// Preset profiles lock to their specific engine
const presetEngine = selectedProfile.preset_engine;
return ENGINE_OPTIONS.filter((opt) => opt.engine === presetEngine);
}

return ENGINE_OPTIONS;
Copy link
Contributor

@coderabbitai coderabbitai bot Mar 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Match available options to the compatibility rules below.

getAvailableOptions() still returns Kokoro for cloned profiles, even though isProfileCompatibleWithEngine() correctly says cloned voices only work with CLONING_ENGINES. Right now the selector can still drive the form into an invalid profile/engine combination.

💡 Keep the dropdown consistent with the helper 
 function getAvailableOptions(selectedProfile?: VoiceProfileResponse | null) {
   if (!selectedProfile) return ENGINE_OPTIONS;
 
   const voiceType = selectedProfile.voice_type || 'cloned';
 
   if (voiceType === 'preset') {
     // Preset profiles lock to their specific engine
     const presetEngine = selectedProfile.preset_engine;
     return ENGINE_OPTIONS.filter((opt) => opt.engine === presetEngine);
   }
 
-  return ENGINE_OPTIONS;
+  return ENGINE_OPTIONS.filter((opt) => CLONING_ENGINES.has(opt.engine));
 }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
function getAvailableOptions(selectedProfile?: VoiceProfileResponse | null) {
if (!selectedProfile) return ENGINE_OPTIONS;
const voiceType = selectedProfile.voice_type || 'cloned';
if (voiceType === 'preset') {
// Preset profiles lock to their specific engine
const presetEngine = selectedProfile.preset_engine;
return ENGINE_OPTIONS.filter((opt) => opt.engine === presetEngine);
}
return ENGINE_OPTIONS;
function getAvailableOptions(selectedProfile?: VoiceProfileResponse | null) {
if (!selectedProfile) return ENGINE_OPTIONS;
const voiceType = selectedProfile.voice_type || 'cloned';
if (voiceType === 'preset') {
// Preset profiles lock to their specific engine
const presetEngine = selectedProfile.preset_engine;
return ENGINE_OPTIONS.filter((opt) => opt.engine === presetEngine);
}
return ENGINE_OPTIONS.filter((opt) => CLONING_ENGINES.has(opt.engine));
}
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@app/src/components/Generation/EngineModelSelector.tsx` around lines 53 - 64,
getAvailableOptions currently returns ENGINE_OPTIONS for cloned profiles causing
invalid profile/engine combos; update getAvailableOptions to filter
ENGINE_OPTIONS using the same compatibility rules as
isProfileCompatibleWithEngine (or explicitly filter by CLONING_ENGINES when
selectedProfile.voice_type === 'cloned'), keep the existing preset branch that
filters by selectedProfile.preset_engine, and return only options whose
opt.engine is in CLONING_ENGINES (or passes
isProfileCompatibleWithEngine(selectedProfile, opt.engine)) so the dropdown
cannot select incompatible engines.

}

function getSelectValue(engine: string, modelSize?: string): string {
if (engine === 'qwen') return `qwen:${modelSize || '1.7B'}`;
if (engine === 'tada') return `tada:${modelSize || '1B'}`;
Expand Down Expand Up @@ -85,12 +114,21 @@ function handleEngineChange(form: UseFormReturn<GenerationFormValues>, value: st
interface EngineModelSelectorProps {
form: UseFormReturn<GenerationFormValues>;
compact?: boolean;
selectedProfile?: VoiceProfileResponse | null;
}

export function EngineModelSelector({ form, compact }: EngineModelSelectorProps) {
export function EngineModelSelector({ form, compact, selectedProfile }: EngineModelSelectorProps) {
const engine = form.watch('engine') || 'qwen';
const modelSize = form.watch('modelSize');
const selectValue = getSelectValue(engine, modelSize);
const availableOptions = getAvailableOptions(selectedProfile);

// If current engine isn't in available options, auto-switch to first available
const currentEngineAvailable = availableOptions.some((opt) => opt.value === selectValue);
if (!currentEngineAvailable && availableOptions.length > 0) {
// Defer to avoid setting state during render
setTimeout(() => handleEngineChange(form, availableOptions[0].value), 0);
}
Comment on lines +111 to +116
Copy link
Contributor

@coderabbitai coderabbitai bot Mar 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Verify render-phase side effect currently exists and useEffect is absent.
rg -n --type=tsx -C3 'setTimeout\(\(\) => handleEngineChange\(form, availableOptions\[0\]\.value\), 0\)' app/src/components/Generation/EngineModelSelector.tsx
rg -n --type=tsx 'useEffect\(' app/src/components/Generation/EngineModelSelector.tsx

Repository: jamiepine/voicebox

Length of output: 117

🏁 Script executed:

fd -name "EngineModelSelector.tsx" app/src/components/

Repository: jamiepine/voicebox

Length of output: 292

🏁 Script executed:

cat -n app/src/components/Generation/EngineModelSelector.tsx | head -130 | tail -50

Repository: jamiepine/voicebox

Length of output: 2341

🏁 Script executed:

rg -n "setTimeout" app/src/components/Generation/EngineModelSelector.tsx

Repository: jamiepine/voicebox

Length of output: 143

🏁 Script executed:

rg -n "useEffect" app/src/components/Generation/EngineModelSelector.tsx

Repository: jamiepine/voicebox

Length of output: 44

Move engine fallback logic to useEffect to avoid side effects during render.

Lines 113–116 schedule a form update via setTimeout during the component render. Side effects should not execute in the render phase—this can cause unpredictable state transitions and issues in React Strict Mode. Move this logic into a useEffect hook:

+import { useEffect } from 'react';
 import type { UseFormReturn } from 'react-hook-form';

@@
-  // If current engine isn't in available options, auto-switch to first available
-  const currentEngineAvailable = availableOptions.some((opt) => opt.value === selectValue);
-  if (!currentEngineAvailable && availableOptions.length > 0) {
-    // Defer to avoid setting state during render
-    setTimeout(() => handleEngineChange(form, availableOptions[0].value), 0);
-  }
+  useEffect(() => {
+    if (!currentEngineAvailable && availableOptions.length > 0) {
+      handleEngineChange(form, availableOptions[0].value);
+    }
+  }, [currentEngineAvailable, availableOptions, form]);
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
// If current engine isn't in available options, auto-switch to first available
const currentEngineAvailable = availableOptions.some((opt) => opt.value === selectValue);
if (!currentEngineAvailable && availableOptions.length > 0) {
// Defer to avoid setting state during render
setTimeout(() => handleEngineChange(form, availableOptions[0].value), 0);
}
import { useEffect } from 'react';
import type { UseFormReturn } from 'react-hook-form';
useEffect(() => {
const currentEngineAvailable = availableOptions.some((opt) => opt.value === selectValue);
if (!currentEngineAvailable && availableOptions.length > 0) {
handleEngineChange(form, availableOptions[0].value);
}
}, [selectValue, availableOptions, form, handleEngineChange]);
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@app/src/components/Generation/EngineModelSelector.tsx` around lines 111 -
116, The engine fallback logic currently performs a side effect during render
(using setTimeout) — move this into a useEffect: compute currentEngineAvailable
from availableOptions and selectValue, then inside a useEffect that depends on
[availableOptions, selectValue, form] call handleEngineChange(form,
availableOptions[0].value) when !currentEngineAvailable and
availableOptions.length > 0; remove the setTimeout-based call from the render
path to avoid state changes during render and React Strict Mode warnings.


const itemClass = compact ? 'text-xs text-muted-foreground' : undefined;
const triggerClass = compact
Expand All @@ -105,7 +143,7 @@ export function EngineModelSelector({ form, compact }: EngineModelSelectorProps)
</SelectTrigger>
</FormControl>
<SelectContent>
{ENGINE_OPTIONS.map((opt) => (
{availableOptions.map((opt) => (
<SelectItem key={opt.value} value={opt.value} className={itemClass}>
{opt.label}
</SelectItem>
Expand All @@ -119,3 +157,17 @@ export function EngineModelSelector({ form, compact }: EngineModelSelectorProps)
export function getEngineDescription(engine: string): string {
return ENGINE_DESCRIPTIONS[engine] ?? '';
}

/**
* Check if a profile is compatible with the currently selected engine.
* Useful for UI hints.
*/
export function isProfileCompatibleWithEngine(
profile: VoiceProfileResponse,
engine: string,
): boolean {
const voiceType = profile.voice_type || 'cloned';
if (voiceType === 'preset') return profile.preset_engine === engine;
if (voiceType === 'cloned') return CLONING_ENGINES.has(engine);
return !PRESET_ONLY_ENGINES.has(engine); // designed — future
}
64 changes: 60 additions & 4 deletions app/src/components/Generation/FloatingGenerateBox.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ export function FloatingGenerateBox({
}: FloatingGenerateBoxProps) {
const selectedProfileId = useUIStore((state) => state.selectedProfileId);
const setSelectedProfileId = useUIStore((state) => state.setSelectedProfileId);
const setSelectedEngine = useUIStore((state) => state.setSelectedEngine);
const { data: selectedProfile } = useProfile(selectedProfileId || '');
const { data: profiles } = useProfiles();
const [isExpanded, setIsExpanded] = useState(false);
Expand Down Expand Up @@ -67,7 +68,12 @@ export function FloatingGenerateBox({
}
},
getEffectsChain: () => {
if (!selectedPresetId || !effectPresets) return undefined;
if (!selectedPresetId) return undefined;
// Profile's own effects chain (no matching preset)
if (selectedPresetId === '_profile') {
return selectedProfile?.effects_chain ?? undefined;
}
if (!effectPresets) return undefined;
const preset = effectPresets.find((p) => p.id === selectedPresetId);
return preset?.effects_chain;
},
Expand Down Expand Up @@ -110,12 +116,56 @@ export function FloatingGenerateBox({
}
}, [selectedProfileId, profiles, setSelectedProfileId]);

// Sync generation form language with selected profile's language
// Sync engine selection to global store so ProfileList can filter
const watchedEngine = form.watch('engine');
useEffect(() => {
if (watchedEngine) {
setSelectedEngine(watchedEngine);
}
}, [watchedEngine, setSelectedEngine]);

// Sync generation form language, engine, and effects with selected profile
useEffect(() => {
if (selectedProfile?.language) {
form.setValue('language', selectedProfile.language as LanguageCode);
}
}, [selectedProfile, form]);
// Auto-switch engine if profile has a default
if (selectedProfile?.default_engine) {
form.setValue(
'engine',
selectedProfile.default_engine as
| 'qwen'
| 'luxtts'
| 'chatterbox'
| 'chatterbox_turbo'
| 'tada'
| 'kokoro',
);
}
// Pre-fill effects from profile defaults
if (
selectedProfile?.effects_chain &&
selectedProfile.effects_chain.length > 0 &&
effectPresets
) {
// Try to match against a known preset
const profileChainJson = JSON.stringify(selectedProfile.effects_chain);
const matchingPreset = effectPresets.find(
(p) => JSON.stringify(p.effects_chain) === profileChainJson,
);
if (matchingPreset) {
setSelectedPresetId(matchingPreset.id);
} else {
// No matching preset — use special value to pass profile chain directly
setSelectedPresetId('_profile');
}
} else if (
selectedProfile &&
(!selectedProfile.effects_chain || selectedProfile.effects_chain.length === 0)
) {
setSelectedPresetId(null);
}
}, [selectedProfile, effectPresets, form]);

// Auto-resize textarea based on content (only when expanded)
useEffect(() => {
Expand Down Expand Up @@ -358,7 +408,7 @@ export function FloatingGenerateBox({
/>

<FormItem className="flex-1 space-y-0">
<EngineModelSelector form={form} compact />
<EngineModelSelector form={form} compact selectedProfile={selectedProfile} />
</FormItem>

<FormItem className="flex-1 space-y-0">
Expand All @@ -375,6 +425,12 @@ export function FloatingGenerateBox({
<SelectItem value="none" className="text-xs">
No effects
</SelectItem>
{selectedProfile?.effects_chain &&
selectedProfile.effects_chain.length > 0 && (
<SelectItem value="_profile" className="text-xs">
Profile default
</SelectItem>
)}
{effectPresets?.map((preset) => (
<SelectItem key={preset.id} value={preset.id} className="text-xs">
{preset.name}
Expand Down
2 changes: 1 addition & 1 deletion app/src/components/Generation/GenerationForm.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,7 @@ export function GenerationForm() {
<div className="grid gap-4 md:grid-cols-3">
<FormItem>
<FormLabel>Model</FormLabel>
<EngineModelSelector form={form} />
<EngineModelSelector form={form} selectedProfile={selectedProfile} />
<FormDescription>
{getEngineDescription(form.watch('engine') || 'qwen')}
</FormDescription>
Expand Down
5 changes: 4 additions & 1 deletion app/src/components/ServerSettings/ModelManagement.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,8 @@ const MODEL_DESCRIPTIONS: Record<string, string> = {
'HumeAI TADA 1B — English speech-language model built on Llama 3.2 1B. Generates 700s+ of coherent audio with synchronized text-acoustic alignment.',
'tada-3b-ml':
'HumeAI TADA 3B Multilingual — built on Llama 3.2 3B. Supports 10 languages with high-fidelity voice cloning via text-acoustic dual alignment.',
kokoro:
'Kokoro 82M by hexgrad. Tiny 82M-parameter TTS that runs at CPU realtime. Supports 8 languages with pre-built voice styles. Apache 2.0 licensed.',
'whisper-base':
'Smallest Whisper model (74M parameters). Fast transcription with moderate accuracy.',
'whisper-small':
Expand Down Expand Up @@ -396,7 +398,8 @@ export function ModelManagement() {
m.model_name.startsWith('qwen-tts') ||
m.model_name.startsWith('luxtts') ||
m.model_name.startsWith('chatterbox') ||
m.model_name.startsWith('tada'),
m.model_name.startsWith('tada') ||
m.model_name.startsWith('kokoro'),
) ?? [];
const whisperModels = modelStatus?.models.filter((m) => m.model_name.startsWith('whisper')) ?? [];

Expand Down
10 changes: 10 additions & 0 deletions app/src/components/VoiceProfiles/ProfileCard.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,16 @@ export function ProfileCard({ profile }: ProfileCardProps) {
<Badge variant="outline" className="text-xs h-5 px-1.5 text-muted-foreground">
{profile.language}
</Badge>
{profile.voice_type === 'preset' && (
<Badge variant="secondary" className="text-xs h-5 px-1.5">
{profile.preset_engine}
</Badge>
)}
{profile.voice_type === 'designed' && (
<Badge variant="secondary" className="text-xs h-5 px-1.5">
designed
</Badge>
)}
{profile.effects_chain && profile.effects_chain.length > 0 && (
<Sparkles className="h-3.5 w-3.5 text-accent fill-accent" />
)}
Expand Down
Loading