Skip to content

[Flutter SDK] Add configuration validation for LLM, STT, and TTS (#450) #456

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
DevDesai444 wants to merge 13 commits into
base: main
Choose a base branch
from
Open

[Flutter SDK] Add configuration validation for LLM, STT, and TTS (#450) #456

Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
e98124d
Add Flutter LLM configuration validation model
DevDesai444 Mar 15, 2026
15138c5
Export Flutter LLM configuration from package root
DevDesai444 Mar 15, 2026
7154800
Add Flutter STT configuration validation model
DevDesai444 Mar 15, 2026
1f85521
Export Flutter STT configuration from package root
DevDesai444 Mar 15, 2026
40829dd
Add Flutter TTS configuration validation
DevDesai444 Mar 15, 2026
01d9b3c
Export Flutter TTS configuration from package root
DevDesai444 Mar 15, 2026
616c961
Validate Flutter LLM settings before FFI calls
DevDesai444 Mar 15, 2026
418f015
Validate Flutter STT settings before transcription
DevDesai444 Mar 15, 2026
115e330
Validate Flutter TTS settings before synthesis
DevDesai444 Mar 15, 2026
8804c3d
Preserve LLM context length for Flutter validation
DevDesai444 Mar 15, 2026
875a9f7
Use loaded LLM context length for Flutter validation
DevDesai444 Mar 17, 2026
86851fe
Make Flutter SDKError rethrows explicit across public APIs
DevDesai444 Mar 17, 2026
1fc42e6
Extract Flutter TTS configuration into a dedicated model file
DevDesai444 Mar 17, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 48 additions & 0 deletions sdk/runanywhere-flutter/packages/runanywhere/lib/features/llm/llm_configuration.dart
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
import 'package:runanywhere/core/protocols/component/component_configuration.dart';
import 'package:runanywhere/core/types/model_types.dart';
import 'package:runanywhere/foundation/error_types/sdk_error.dart';

/// Configuration for the LLM component.
///
/// Mirrors the validation contract used by the Swift and Kotlin SDKs so
/// invalid parameters fail in Dart before crossing the FFI boundary.
class LLMConfiguration implements ComponentConfiguration {
final String? modelId;
final InferenceFramework? preferredFramework;
final int contextLength;
final double temperature;
final int maxTokens;
final String? systemPrompt;
final bool streamingEnabled;

const LLMConfiguration({
this.modelId,
this.preferredFramework,
this.contextLength = 2048,
this.temperature = 0.7,
this.maxTokens = 100,
this.systemPrompt,
this.streamingEnabled = true,
});

@override
void validate() {
if (contextLength <= 0 || contextLength > 32768) {
throw SDKError.validationFailed(
'Context length must be between 1 and 32768',
);
}

if (!temperature.isFinite || temperature < 0 || temperature > 2.0) {
throw SDKError.validationFailed(
'Temperature must be between 0 and 2.0',
);
}

if (maxTokens <= 0 || maxTokens > contextLength) {
throw SDKError.validationFailed(
'Max tokens must be between 1 and context length',
);
}
}
}
46 changes: 46 additions & 0 deletions sdk/runanywhere-flutter/packages/runanywhere/lib/features/stt/stt_configuration.dart
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
import 'package:runanywhere/core/protocols/component/component_configuration.dart';
import 'package:runanywhere/core/types/model_types.dart';
import 'package:runanywhere/foundation/error_types/sdk_error.dart';

/// Configuration for the STT component.
///
/// Mirrors the validation contract used by the Swift and Kotlin SDKs so
/// invalid parameters fail in Dart before crossing the FFI boundary.
class STTConfiguration implements ComponentConfiguration {
final String? modelId;
final InferenceFramework? preferredFramework;
final String language;
final int sampleRate;
final bool enablePunctuation;
final bool enableDiarization;
final List<String> vocabularyList;
final int maxAlternatives;
final bool enableTimestamps;

const STTConfiguration({
this.modelId,
this.preferredFramework,
this.language = 'en-US',
this.sampleRate = 16000,
this.enablePunctuation = true,
this.enableDiarization = false,
this.vocabularyList = const <String>[],
this.maxAlternatives = 1,
this.enableTimestamps = true,
});

@override
void validate() {
if (sampleRate <= 0 || sampleRate > 48000) {
throw SDKError.validationFailed(
'Sample rate must be between 1 and 48000 Hz',
);
}

if (maxAlternatives <= 0 || maxAlternatives > 10) {
throw SDKError.validationFailed(
'Max alternatives must be between 1 and 10',
);
}
}
}
25 changes: 24 additions & 1 deletion sdk/runanywhere-flutter/packages/runanywhere/lib/features/tts/system_tts_service.dart
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,11 @@ import 'dart:async';
import 'dart:typed_data';

import 'package:flutter_tts/flutter_tts.dart';
import 'package:runanywhere/core/protocols/component/component_configuration.dart';
import 'package:runanywhere/foundation/error_types/sdk_error.dart';

/// Configuration for TTS synthesis
class TTSConfiguration {
class TTSConfiguration implements ComponentConfiguration {
final String voice;
final String language;
final double speakingRate;
Expand All @@ -26,6 +28,27 @@ class TTSConfiguration {
this.volume = 1.0,
this.audioFormat = 'pcm',
});

@override
void validate() {
if (!speakingRate.isFinite || speakingRate < 0.5 || speakingRate > 2.0) {
throw SDKError.validationFailed(
'Speaking rate must be between 0.5 and 2.0',
);
}

if (!pitch.isFinite || pitch < 0.5 || pitch > 2.0) {
throw SDKError.validationFailed(
'Pitch must be between 0.5 and 2.0',
);
}

if (!volume.isFinite || volume < 0.0 || volume > 1.0) {
throw SDKError.validationFailed(
'Volume must be between 0.0 and 1.0',
);
}
}
}

/// Input for TTS synthesis
Expand Down
30 changes: 29 additions & 1 deletion sdk/runanywhere-flutter/packages/runanywhere/lib/native/dart_bridge_llm.dart
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ import 'dart:ffi';
import 'dart:isolate'; // Keep for non-streaming generation

import 'package:ffi/ffi.dart';

import 'package:runanywhere/features/llm/llm_configuration.dart';
import 'package:runanywhere/foundation/logging/sdk_logger.dart';
import 'package:runanywhere/native/ffi_types.dart';
import 'package:runanywhere/native/platform_loader.dart';
Expand Down Expand Up @@ -241,6 +241,12 @@ class DartBridgeLLM {
double temperature = 0.7,
String? systemPrompt,
}) async {
_validateGenerationParameters(
maxTokens: maxTokens,
temperature: temperature,
systemPrompt: systemPrompt,
);

final handle = getHandle();

if (!isLoaded) {
Expand Down Expand Up @@ -284,6 +290,13 @@ class DartBridgeLLM {
double temperature = 0.7,
String? systemPrompt,
}) {
_validateGenerationParameters(
maxTokens: maxTokens,
temperature: temperature,
systemPrompt: systemPrompt,
streamingEnabled: true,
);

final handle = getHandle();

if (!isLoaded) {
Expand Down Expand Up @@ -367,6 +380,21 @@ class DartBridgeLLM {
}
}

void _validateGenerationParameters({
required int maxTokens,
required double temperature,
String? systemPrompt,
bool streamingEnabled = false,
}) {
LLMConfiguration(
contextLength: 32768,
maxTokens: maxTokens,
temperature: temperature,
systemPrompt: systemPrompt,
streamingEnabled: streamingEnabled,
).validate();
}
Comment on lines +401 to +415
Copy link
Copy Markdown
Contributor

@greptile-apps greptile-apps bot Mar 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hardcoded contextLength defeats maxTokens validation

The contextLength is hardcoded to 32768 (the maximum allowed), which means the maxTokens <= contextLength check in LLMConfiguration.validate() will never reject any value under 32768. A user could pass maxTokens: 32768 even though the loaded model may have a much smaller context window (e.g. 2048 or 4096), sending an invalid value across the FFI boundary — exactly what this PR is trying to prevent.

Consider passing the actual model's context length here. Since DartBridgeLLM manages the C++ lifecycle, the real context length may be queryable from the native layer, or you could store it when the model is loaded and pass it through to validation.

Prompt To Fix With AI 
This is a comment left during a code review.
Path: sdk/runanywhere-flutter/packages/runanywhere/lib/native/dart_bridge_llm.dart
Line: 383-396

Comment:
**Hardcoded `contextLength` defeats `maxTokens` validation**

The `contextLength` is hardcoded to `32768` (the maximum allowed), which means the `maxTokens <= contextLength` check in `LLMConfiguration.validate()` will never reject any value under 32768. A user could pass `maxTokens: 32768` even though the loaded model may have a much smaller context window (e.g. 2048 or 4096), sending an invalid value across the FFI boundary — exactly what this PR is trying to prevent.

Consider passing the actual model's context length here. Since `DartBridgeLLM` manages the C++ lifecycle, the real context length may be queryable from the native layer, or you could store it when the model is loaded and pass it through to validation.

How can I resolve this? If you propose a fix, please make it concise.


// MARK: - Cleanup

/// Destroy the component and release resources.
Expand Down
4 changes: 3 additions & 1 deletion sdk/runanywhere-flutter/packages/runanywhere/lib/native/dart_bridge_stt.dart
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ import 'dart:isolate';
import 'dart:typed_data';

import 'package:ffi/ffi.dart';

import 'package:runanywhere/features/stt/stt_configuration.dart';
import 'package:runanywhere/foundation/logging/sdk_logger.dart';
import 'package:runanywhere/native/ffi_types.dart';
import 'package:runanywhere/native/platform_loader.dart';
Expand Down Expand Up @@ -185,6 +185,8 @@ class DartBridgeSTT {
Uint8List audioData, {
int sampleRate = 16000,
}) async {
STTConfiguration(sampleRate: sampleRate).validate();

final handle = getHandle();

if (!isLoaded) {
Expand Down
9 changes: 8 additions & 1 deletion sdk/runanywhere-flutter/packages/runanywhere/lib/native/dart_bridge_tts.dart
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,8 @@ import 'dart:isolate';
import 'dart:typed_data';

import 'package:ffi/ffi.dart';

import 'package:runanywhere/features/tts/system_tts_service.dart'
show TTSConfiguration;
import 'package:runanywhere/foundation/logging/sdk_logger.dart';
import 'package:runanywhere/native/ffi_types.dart';
import 'package:runanywhere/native/platform_loader.dart';
Expand Down Expand Up @@ -190,6 +191,12 @@ class DartBridgeTTS {
double pitch = 1.0,
double volume = 1.0,
}) async {
TTSConfiguration(
speakingRate: rate,
pitch: pitch,
volume: volume,
).validate();

final handle = getHandle();

if (!isLoaded) {
Expand Down
2 changes: 2 additions & 0 deletions sdk/runanywhere-flutter/packages/runanywhere/lib/public/runanywhere.dart
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1844,6 +1844,8 @@ class RunAnywhere {
tokensPerSecond: tokensPerSecond,
structuredData: structuredData,
);
} on SDKError {
rethrow;
Comment on lines +1856 to +1857
Copy link
Copy Markdown
Contributor

@greptile-apps greptile-apps bot Mar 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing parallel SDKError rethrow in generateStream, transcribe, and synthesize

The on SDKError { rethrow; } clause was added here for generate, but the sibling generateStream method (line ~1930) calls DartBridge.llm.generateStream() outside any try-catch. While this works because validation errors propagate naturally (they're thrown synchronously before the stream is returned), it creates an inconsistency in error handling patterns.

More importantly, transcribe (line 684) and synthesize (line 947) both have a generic catch (e) { ... rethrow; } which does preserve the SDKError via rethrow. This means the generate method was the only place actively swallowing SDKError (via throw SDKError.generationFailed('$e')). This fix is correct — just calling it out for completeness that only generate had this masking issue.

Prompt To Fix With AI 
This is a comment left during a code review.
Path: sdk/runanywhere-flutter/packages/runanywhere/lib/public/runanywhere.dart
Line: 1847-1848

Comment:
**Missing parallel `SDKError` rethrow in `generateStream`, `transcribe`, and `synthesize`**

The `on SDKError { rethrow; }` clause was added here for `generate`, but the sibling `generateStream` method (line ~1930) calls `DartBridge.llm.generateStream()` outside any try-catch. While this works because validation errors propagate naturally (they're thrown synchronously before the stream is returned), it creates an inconsistency in error handling patterns.

More importantly, `transcribe` (line 684) and `synthesize` (line 947) both have a generic `catch (e) { ... rethrow; }` which *does* preserve the `SDKError` via `rethrow`. This means the `generate` method was the only place actively swallowing `SDKError` (via `throw SDKError.generationFailed('$e')`). This fix is correct — just calling it out for completeness that only `generate` had this masking issue.

How can I resolve this? If you propose a fix, please make it concise.

} catch (e) {
// Track generation failure
TelemetryService.shared.trackError(
Expand Down
3 changes: 3 additions & 0 deletions sdk/runanywhere-flutter/packages/runanywhere/lib/runanywhere.dart
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,9 @@ export 'core/types/sdk_component.dart';
export 'core/types/storage_types.dart';
// Network layer
export 'data/network/network.dart';
export 'features/llm/llm_configuration.dart';
export 'features/stt/stt_configuration.dart';
export 'features/tts/system_tts_service.dart' show TTSConfiguration;
export 'features/vad/vad_configuration.dart';
export 'foundation/configuration/sdk_constants.dart';
export 'foundation/error_types/sdk_error.dart';
Expand Down