Hybrid routing: local-first STT with cloud fallback#467
Open
Siddhesh2377 wants to merge 11 commits intomainfrom
Open
Hybrid routing: local-first STT with cloud fallback#467Siddhesh2377 wants to merge 11 commits intomainfrom
Siddhesh2377 wants to merge 11 commits intomainfrom
Conversation
- Add RAC_FRAMEWORK_SARVAM enum and RAC_ERROR_AUDIO_TOO_LONG error code - Implement sarvam backend with STT vtable (WAV encoding, multipart HTTP, JSON parsing) - Register as low-priority (10) provider for cloud fallback - Add 14 unit/integration tests with mock HTTP executor - Add RAC_BACKEND_SARVAM CMake option (default: OFF)
- Add SAARIKA_V2_5 model variant (v2 is deprecated by Sarvam) - Default model is now saarika:v2.5 - Add curl-based HTTP executor for real API integration tests - Fix lifecycle_manager.cpp missing condition_variable include (GCC 15) - All 15 tests passing including real Sarvam API transcription
- Add SARVAM("Sarvam") enum variant with displayName and analyticsKey
- STT bridge is framework-agnostic, no other changes needed
- Add 11 unit tests for SARVAM InferenceFramework enum and STT types - Add SARVAM=10 to CppBridgeModelRegistry.Framework constants - Fix exhaustive when expressions in ModelManagement for SARVAM - Remove broken pre-existing SDKTest.kt
Wire Sarvam AI cloud STT through C++ JNI layer. Fix language_code parsing in JNI bridge that caused 400 errors. Add HTTP executor support and Sarvam Kotlin bridge.
Router selects local backend first, cascades to cloud when confidence is below 0.5 threshold. Backends declare their own routing conditions. Whisper model restored after cloud fallback.
Android uses ConnectivityManager, JVM returns true. Removes brittle reflection from jvmAndroidMain.
Show backend name, confidence score, and fallback status in STT screen. Remove Sarvam from model picker. Remove Cloud STT screen from navigation — routing handles cloud fallback automatically.
9 unit tests (commonTest) + 5 instrumented tests (device). Add docs/impl/hybrid-routing.md covering cascade logic, API key setup, language mapping, and how to add new providers.
📝 WalkthroughWalkthroughAdds a hybrid STT routing system (local-first with confidence-based cloud fallback), new Sarvam cloud STT backend (C++ + JNI + Kotlin), an HTTP executor bridge between C++ and Kotlin, Kotlin routing primitives/registry, sample app UI/VM wiring, build/test additions, and documentation updates. Changes
Sequence Diagram(s)mermaid Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related PRs
Suggested labels
Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
...ples/android/RunAnywhereAI/app/src/main/java/com/runanywhere/runanywhereai/data/ModelList.kt
Show resolved
Hide resolved
...rc/jvmAndroidMain/kotlin/com/runanywhere/sdk/public/extensions/RunAnywhere+STT.jvmAndroid.kt
Show resolved
Hide resolved
...where-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/backends/stt/WhisperSTTBackend.kt
Show resolved
Hide resolved
...nywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/routing/HybridRouterRegistry.kt
Outdated
Show resolved
Hide resolved
...ywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/backends/stt/SarvamSTTBackend.kt
Show resolved
Hide resolved
Collaborator
Author
With Cloud Fallback :
With Local Model :
cc: @sanchitmonga22 |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 19
Note
Due to the large number of review comments, Critical, Major severity comments were prioritized as inline comments.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
sdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/public/extensions/RunAnywhere+ModelManagement.jvmAndroid.kt (1)
94-103:⚠️ Potential issue | 🟠 MajorSARVAM mapping is currently one-way; read paths still degrade to
UNKNOWN.You added SARVAM in the write/extraction mappings (Lines 102 and 718), but the same file still misses SARVAM in
bridgeModelToPublic(...)and framework-int parsing. That causes framework loss after registry/assignment reads.🔧 Proposed follow-up patch (outside changed lines)
@@ framework = when (bridge.framework) { CppBridgeModelRegistry.Framework.LLAMACPP -> InferenceFramework.LLAMA_CPP CppBridgeModelRegistry.Framework.ONNX -> InferenceFramework.ONNX CppBridgeModelRegistry.Framework.FOUNDATION_MODELS -> InferenceFramework.FOUNDATION_MODELS CppBridgeModelRegistry.Framework.SYSTEM_TTS -> InferenceFramework.SYSTEM_TTS + CppBridgeModelRegistry.Framework.FLUID_AUDIO -> InferenceFramework.FLUID_AUDIO + CppBridgeModelRegistry.Framework.BUILTIN -> InferenceFramework.BUILT_IN + CppBridgeModelRegistry.Framework.NONE -> InferenceFramework.NONE + CppBridgeModelRegistry.Framework.SARVAM -> InferenceFramework.SARVAM else -> InferenceFramework.UNKNOWN }, @@ framework = when (frameworkInt) { - 1 -> InferenceFramework.LLAMA_CPP - 2 -> InferenceFramework.ONNX - 3 -> InferenceFramework.FOUNDATION_MODELS - 4 -> InferenceFramework.SYSTEM_TTS + 0 -> InferenceFramework.ONNX + 1 -> InferenceFramework.LLAMA_CPP + 2 -> InferenceFramework.FOUNDATION_MODELS + 3 -> InferenceFramework.SYSTEM_TTS + 4 -> InferenceFramework.FLUID_AUDIO + 5 -> InferenceFramework.BUILT_IN + 6 -> InferenceFramework.NONE + 10 -> InferenceFramework.SARVAM else -> InferenceFramework.UNKNOWN },Also applies to: 710-719
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@sdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/public/extensions/RunAnywhere`+ModelManagement.jvmAndroid.kt around lines 94 - 103, The SARVAM framework mapping is only handled in the write path causing reads to downgrade to UNKNOWN; update the read/path mappings to include SARVAM too: in bridgeModelToPublic (and any framework-int parsing function used when converting CppBridgeModelRegistry.Framework back to InferenceFramework) add the branch mapping CppBridgeModelRegistry.Framework.SARVAM <-> InferenceFramework.SARVAM so modelInfo.framework and the registry enum round-trip without losing SARVAM.sdk/runanywhere-commons/scripts/build-android.sh (1)
347-360:⚠️ Potential issue | 🟠 MajorDisable RAG and set platform-off flags in this Android CMake invocation.
Line 358 currently enables RAG (
-DRAC_BACKEND_RAG=ON), which conflicts with the repository’s Linux build constraint for commons and can break full backend builds.Proposed flag update
- -DRAC_BACKEND_RAG=ON \ + -DRAC_BACKEND_RAG=OFF \ + -DRAC_BUILD_PLATFORM=OFF \ -DRAC_BACKEND_SARVAM=ON \As per coding guidelines: "Disable RAG backend with
-DRAC_BACKEND_RAG=OFF..." and "Pass-DRAC_BUILD_PLATFORM=OFFwhen running CMake on Linux."🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@sdk/runanywhere-commons/scripts/build-android.sh` around lines 347 - 360, In the CMake invocation block where cmake is run to configure Android builds, change the RAG backend flag from -DRAC_BACKEND_RAG=ON to -DRAC_BACKEND_RAG=OFF and add the platform-off flag -DRAC_BUILD_PLATFORM=OFF to the argument list so RAG is disabled and platform-specific build components are turned off; update the cmake command that contains -DRAC_BACKEND_RAG and the surrounding flags (e.g., -DRAC_BUILD_BACKENDS, -DRAC_BUILD_JNI) to include -DRAC_BUILD_PLATFORM=OFF and set -DRAC_BACKEND_RAG=OFF.
🟡 Minor comments (6)
examples/android/RunAnywhereAI/app/src/main/java/com/runanywhere/runanywhereai/presentation/stt/SpeechToTextViewModel.kt-613-616 (1)
613-616:⚠️ Potential issue | 🟡 MinorReset routing metadata when a session starts or fails.
These fields are only written on the success path, so a previous backend/fallback banner will survive into the next recording and after failed transcriptions. Please clear them alongside
transcription/metricswhen starting a new capture, clearing results, or handling an error.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@examples/android/RunAnywhereAI/app/src/main/java/com/runanywhere/runanywhereai/presentation/stt/SpeechToTextViewModel.kt` around lines 613 - 616, Reset the routing metadata fields routingBackendId, routingBackendName, wasFallback, and primaryConfidence whenever you clear transcription/metrics or begin a new session: update the places that clear transcription/metrics (e.g., startCapture()/startRecording(), clearResults()/clearTranscription(), and the error path such as handleTranscriptionError()/onTranscriptionFailed()) to also set routingBackendId and routingBackendName to null (or empty), wasFallback to false, and primaryConfidence to a neutral value (e.g., 0.0), so stale backend/fallback banners cannot persist across recordings or after failures.sdk/runanywhere-commons/src/backends/sarvam/README.md-118-122 (1)
118-122:⚠️ Potential issue | 🟡 MinorAdd a language tag to the architecture code fence.
Static analysis is already flagging this block. Using
text/plaintexthere will keep markdown lint green.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@sdk/runanywhere-commons/src/backends/sarvam/README.md` around lines 118 - 122, The fenced code block listing files (the triple-backtick block containing "rac_stt_sarvam.h", "rac_stt_sarvam.cpp", "rac_backend_sarvam_register.cpp") needs a language tag to satisfy markdown linting; change the opening ``` to ```text or ```plaintext in the README.md so the block is treated as plain text and static analysis warnings are resolved.docs/impl/hybrid-routing.md-23-32 (1)
23-32:⚠️ Potential issue | 🟡 MinorAdd language identifiers to these fenced blocks.
markdownlint is already flagging the fences starting on Lines 23, 86, and 162 because they omit a language tag.
Also applies to: 86-92, 162-196
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@docs/impl/hybrid-routing.md` around lines 23 - 32, The fenced ASCII flow diagrams (e.g., the block starting with "Record full audio" and the other similar diagrams further down) lack language identifiers and trigger markdownlint; update each triple-backtick fence to include a language tag such as ```text (or ```txt) so markdownlint stops flagging them, and apply the same fix to the other two fenced blocks containing the ASCII diagrams/reflow (the ones used for the Whisper->Sarvam routing examples) to ensure consistency.docs/impl/hybrid-routing.md-177-179 (1)
177-179:⚠️ Potential issue | 🟡 MinorUpdate the network-check note to match the implementation.
This location note says
NetworkAvailability.ktuses Android reflection, but the implementation in this PR moved network checks toexpect/actual. The doc is already stale.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@docs/impl/hybrid-routing.md` around lines 177 - 179, The doc note about NetworkAvailability.kt is stale: update the hybrid-routing.md section that references NetworkAvailability.kt to reflect that network checks are implemented using Kotlin expect/actual rather than Android reflection; remove the mention of Android reflection and add a brief line indicating that platform-specific implementations live in expect/actual declarations (e.g., the NetworkAvailability expect in common and actual implementations on Android/iOS), and update any file list or path comments to point to the expect/actual implementation locations instead of implying reflection-based behavior.sdk/runanywhere-kotlin/src/androidInstrumentedTest/kotlin/com/runanywhere/sdk/routing/STTRoutingInstrumentedTest.kt-19-20 (1)
19-20:⚠️ Potential issue | 🟡 MinorUse the SDK wrapper script in the run instructions.
The inline command points contributors at raw Gradle, but this repo standardizes Kotlin SDK operations through
./scripts/sdk.sh. As per coding guidelines, "sdk/runanywhere-kotlin/**: Use the build script./scripts/sdk.shfor all Kotlin Multiplatform SDK operations instead of direct Gradle commands."🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@sdk/runanywhere-kotlin/src/androidInstrumentedTest/kotlin/com/runanywhere/sdk/routing/STTRoutingInstrumentedTest.kt` around lines 19 - 20, Replace the inline Gradle instruction that currently shows "./gradlew :connectedAndroidTest" in the file header comment with the repository-standard SDK wrapper invocation (e.g. "./scripts/sdk.sh :connectedAndroidTest"); update the comment line containing the exact string "./gradlew :connectedAndroidTest" so contributors use the SDK wrapper script instead of raw Gradle.sdk/runanywhere-commons/src/backends/sarvam/rac_stt_sarvam.cpp-132-136 (1)
132-136:⚠️ Potential issue | 🟡 MinorThread-safety issue: Returned pointer may be invalidated after mutex release.
rac_stt_sarvam_get_api_key()returnskey.c_str()after releasing the mutex. If another thread callsrac_stt_sarvam_set_api_key(), the underlyingstd::stringmay reallocate, invalidating the returned pointer. This creates a potential use-after-free.Since this function is primarily used for null/non-null checks (e.g.,
SarvamBridge.nativeHasApiKey()), consider:
- Returning a copy (caller must free), or
- Adding a
rac_stt_sarvam_has_api_key()function that returns bool🛡️ Option: Add a safe has_api_key function
+RAC_SARVAM_API rac_bool_t rac_stt_sarvam_has_api_key(void) { + std::lock_guard<std::mutex> lock(rac::sarvam::global_api_key_mutex()); + return rac::sarvam::global_api_key().empty() ? RAC_FALSE : RAC_TRUE; +} + const char* rac_stt_sarvam_get_api_key(void) { std::lock_guard<std::mutex> lock(rac::sarvam::global_api_key_mutex()); const auto& key = rac::sarvam::global_api_key(); return key.empty() ? nullptr : key.c_str(); }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@sdk/runanywhere-commons/src/backends/sarvam/rac_stt_sarvam.cpp` around lines 132 - 136, The current rac_stt_sarvam_get_api_key() returns key.c_str() after releasing the mutex which can be invalidated; change the API to avoid returning a dangling pointer by either (A) implementing rac_stt_sarvam_has_api_key() that acquires the rac::sarvam::global_api_key_mutex(), checks !global_api_key().empty(), returns a bool and update callers such as SarvamBridge.nativeHasApiKey() to use it, or (B) make rac_stt_sarvam_get_api_key() return a heap-allocated copy (caller must free) by locking the mutex, copying the string into a newly allocated char* and returning that pointer and document the ownership, and ensure rac_stt_sarvam_set_api_key() still protects writes with rac::sarvam::global_api_key_mutex(); prefer option A for minimal churn.
🧹 Nitpick comments (6)
sdk/runanywhere-commons/scripts/build-android.sh (1)
589-593: Harden Sarvam.socopy path handling.Line 590 checks only one output path and silently skips on miss. Add fallback path + warning (consistent with other backend copy logic) to avoid packaging gaps.
Suggested copy-path hardening
- if [ -f "${ABI_BUILD_DIR}/src/backends/sarvam/librac_backend_sarvam.so" ]; then - cp "${ABI_BUILD_DIR}/src/backends/sarvam/librac_backend_sarvam.so" "${JNI_DIST_DIR}/${ABI}/" - echo " Copied: librac_backend_sarvam.so -> jni/${ABI}/" - fi + if [ -f "${ABI_BUILD_DIR}/src/backends/sarvam/librac_backend_sarvam.so" ]; then + cp "${ABI_BUILD_DIR}/src/backends/sarvam/librac_backend_sarvam.so" "${JNI_DIST_DIR}/${ABI}/" + echo " Copied: librac_backend_sarvam.so -> jni/${ABI}/" + elif [ -f "${ABI_BUILD_DIR}/backends/sarvam/librac_backend_sarvam.so" ]; then + cp "${ABI_BUILD_DIR}/backends/sarvam/librac_backend_sarvam.so" "${JNI_DIST_DIR}/${ABI}/" + echo " Copied: librac_backend_sarvam.so -> jni/${ABI}/" + else + print_warning "librac_backend_sarvam.so not found for ${ABI}" + fi🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@sdk/runanywhere-commons/scripts/build-android.sh` around lines 589 - 593, The current copy for sarvam .so only checks "${ABI_BUILD_DIR}/src/backends/sarvam/librac_backend_sarvam.so" and silently skips if missing; update the block around that check to try a fallback path (e.g. "${ABI_BUILD_DIR}/backends/sarvam/librac_backend_sarvam.so") and copy from it if present, and if neither path exists emit a warning message consistent with other backend copy logic (use ABI_BUILD_DIR, JNI_DIST_DIR, ABI and the filename librac_backend_sarvam.so in the messages).CLAUDE.md (1)
330-330: Use a concrete path instead of ellipsis in the pattern reference.Line 330 uses
.../routing/, which is not directly navigable in many editors. Prefer the full path for quicker lookup.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@CLAUDE.md` at line 330, Replace the non-navigable ellipsis in the pattern string "'.../routing/'" with the concrete directory/package path used in the repository for the runanywhere-kotlin routing module (i.e., the full commonMain routing package path referenced elsewhere), so readers can click/navigate directly; update the CLAUDE.md sentence that currently shows "sdk/runanywhere-kotlin/src/commonMain/.../routing/" to use that full path instead of the ellipsis.sdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/cloud/sarvam/Sarvam.kt (1)
71-76: The-20error code check is dead code and misleading.Per
rac_backend_sarvam_jni.cpp(lines 25-39), the JNI wrapper already handlesRAC_ERROR_MODULE_ALREADY_REGISTEREDinternally and returnsRAC_SUCCESS(0) in that case. The native C++ code actually returns-401for this error, not-20. Since the JNI normalizes it, this check will never trigger.The code works correctly because when already registered,
nativeRegister()returns 0, but the comment is inaccurate and the-20check is confusing.♻️ Suggested simplification
// Register backend val result = SarvamBridge.nativeRegister() - if (result != 0 && result != -20) { // -20 = RAC_ERROR_MODULE_ALREADY_REGISTERED + if (result != 0) { logger.error("Sarvam registration failed: $result") return }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@sdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/cloud/sarvam/Sarvam.kt` around lines 71 - 76, The conditional checking for -20 is dead/misleading; update the Sarvam registration logic by removing the -20 branch and treating any non-zero result from SarvamBridge.nativeRegister() as a failure: call logger.error("Sarvam registration failed: $result") and return when result != 0. Also remove or correct the comment that references -20 and mention that the JNI wrapper normalizes the already-registered case (nativeRegister() returns 0 for already-registered).sdk/runanywhere-kotlin/src/commonMain/kotlin/com/runanywhere/sdk/routing/HybridRouter.kt (1)
96-101: Minor:computeScore()is invoked twice per candidate during logging.The score is computed once for sorting and again in the debug log. For typical small candidate lists this is negligible, but could be optimized by caching scores during sort.
♻️ Optional: Cache scores to avoid recomputation
// Step 3: score and sort - return candidates.sortedByDescending { computeScore(it, context) }.also { sorted -> + val scored = candidates.map { it to computeScore(it, context) } + .sortedByDescending { it.second } + return scored.map { it.first }.also { sorted -> logger.debug( "${capability.name} routing order: " + - sorted.joinToString { "'${it.moduleId}'(${computeScore(it, context)})" } + scored.joinToString { "'${it.first.moduleId}'(${it.second})" } ) }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@sdk/runanywhere-kotlin/src/commonMain/kotlin/com/runanywhere/sdk/routing/HybridRouter.kt` around lines 96 - 101, The code calls computeScore(it, context) twice per candidate (once for sorting and again inside the debug log); change the implementation in HybridRouter to compute and cache each candidate's score once (e.g., map candidates to pairs of (candidate, score) or a small data class), sort by the cached score, then produce the sorted candidate list and use the cached score values in the logger.debug message (referencing computeScore, capability, logger.debug and the sorted candidate list) so scores are not recomputed.sdk/runanywhere-commons/tests/test_stt_sarvam.cpp (2)
524-529: Large memory allocation: 180 seconds of audio at 16kHz.This generates ~5.76 MB of PCM data (180 × 16000 × 2 bytes). While acceptable for testing the "audio too long" path, it may be slow or problematic in constrained CI environments. Consider using a smaller duration that still exceeds the 2-minute limit (e.g., 121 seconds).
♻️ Reduce test audio duration
- // 3 minutes of audio (exceeds 2 min limit) - auto pcm = generate_pcm_sine(440.0f, 180.0f); + // Just over 2 minutes (121s exceeds 120s limit) + auto pcm = generate_pcm_sine(440.0f, 121.0f);🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@sdk/runanywhere-commons/tests/test_stt_sarvam.cpp` around lines 524 - 529, The test currently allocates 180s of PCM via generate_pcm_sine(440.0f, 180.0f) which produces ~5.76MB and can slow CI; change the duration to a minimal value just over the 2-minute limit (e.g., 121 seconds) so that the rac_stt_sarvam_transcribe call still triggers the "audio too long" error and the ASSERT_TRUE(rc != RAC_SUCCESS, ...) assertion remains valid; update the call site where generate_pcm_sine is invoked and ensure pcm.size() usage and the rac_stt_sarvam_transcribe invocation remain unchanged.
289-296: Using0x1as a fake handle may cause undefined behavior if dereferenced.Lines 291 and 295 cast arbitrary values to
rac_handle_t. If the implementation dereferences these before null-checking parameters, this could crash or produce UB. This is acceptable for testing early-validation paths, but fragile if implementation order changes.Consider using a properly allocated but uninitialized struct if the tests need to go deeper, or ensure the implementation always validates other params before dereferencing the handle.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@sdk/runanywhere-commons/tests/test_stt_sarvam.cpp` around lines 289 - 296, The test uses an arbitrary cast (0x1) to rac_handle_t when calling rac_stt_sarvam_transcribe, which can cause undefined behavior if the implementation dereferences the handle; replace the bogus handle with a benign valid handle value (e.g., construct a local rac_handle_t fake_handle/zero-initialized struct or use an existing RAC_INVALID_HANDLE/allocator helper) and pass &fake_handle to rac_stt_sarvam_transcribe in the two null-parameter assertions so the test exercises null-audio/null-result validation without risking handle dereference UB.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In
`@examples/android/RunAnywhereAI/app/src/main/java/com/runanywhere/runanywhereai/data/ModelList.kt`:
- Around line 193-205: The file currently hardcodes a Sarvam API key in the
Sarvam.register(...) call; remove the literal string passed to apiKey and change
Sarvam registration to read the key from a non-committed source (e.g., local
config, BuildConfig field injected at build time, Android keystore, or a secure
server token exchange) and fail gracefully if the key is missing; update the
usage around Sarvam.register(...) and any initialization logic that depends on
it (the RunAnywhere.registerModel(...) block can remain but only after verifying
a valid key was obtained), and ensure logging does not print secrets.
In
`@examples/android/RunAnywhereAI/app/src/main/java/com/runanywhere/runanywhereai/presentation/stt/SpeechToTextScreen.kt`:
- Around line 1111-1115: The UI currently uses the boolean wasFallback to decide
label and color, which is incorrect; update the UI to use explicit backend
metadata (e.g., a BackendSource enum or a field on the recognition result such
as result.backend or result.source) instead of wasFallback when rendering the
text and color in the composable that sets text = if (wasFallback) "Cloud
Fallback" else "Local"; change that logic to switch on the structured backend
enum (e.g., BackendSource.LOCAL, BackendSource.CLOUD) and map to the label
("Local", "Cloud", "Cloud Fallback") and AppColors (AppColors.primaryGreen,
AppColors.primaryOrange) accordingly, and ensure the producer code that creates
recognition results populates this backend field so the UI reads a typed value
rather than relying on wasFallback.
In
`@examples/android/RunAnywhereAI/app/src/main/java/com/runanywhere/runanywhereai/presentation/stt/SpeechToTextViewModel.kt`:
- Around line 621-624: The current Timber.i(...) call in
SpeechToTextViewModel.kt logs the raw transcript text (variable text) which can
expose PII; update the logging in the batch transcription completion path (the
Timber.i invocation that references sttResult.routingBackendId, inferenceTimeMs,
sttResult.confidence, sttResult.wasFallback) to remove the raw transcript body
and only include safe metadata such as routingBackendId, inferenceTimeMs,
confidence, and fallback flag (and consider redacting or hashing any identifiers
if needed), ensuring no variable that contains the full user transcript (e.g.,
text or sttResult.transcript) is written to logs.
In `@sdk/runanywhere-commons/src/backends/sarvam/CMakeLists.txt`:
- Around line 30-38: The CMake logic can create a STATIC rac_backend_sarvam when
RAC_BUILD_JNI is ON, but SarvamBridge.kt expects a shared lib via
System.loadLibrary("rac_backend_sarvam"); update the CMake so that when
RAC_BUILD_JNI is true you force or require a SHARED target: either
set(RAC_BUILD_SHARED ON) when RAC_BUILD_JNI is ON before add_library, or emit a
message(FATAL_ERROR ...) if RAC_BUILD_JNI AND NOT RAC_BUILD_SHARED; ensure the
add_library(rac_backend_sarvam SHARED ${SARVAM_BACKEND_SOURCES}) path is used
for JNI builds so the JVM can load the library.
In `@sdk/runanywhere-commons/src/backends/sarvam/jni/rac_backend_sarvam_jni.cpp`:
- Around line 53-79: nativeSetApiKey currently accepts empty strings and
nativeHasApiKey only checks for non-null, so treat blank API keys as missing by
rejecting empty strings and reporting "no key" when empty: in nativeSetApiKey
after env->GetStringUTFChars(apiKey, nullptr) verify the C string is non-empty
(e.g., key[0] != '\0' or strlen(key) > 0); if empty, call
env->ReleaseStringUTFChars(apiKey, key) and return RAC_ERROR_INVALID_ARGUMENT
instead of proceeding to rac_stt_sarvam_set_api_key. In nativeHasApiKey, replace
the null-only check with a non-empty check against rac_stt_sarvam_get_api_key()
(ensure the returned pointer is non-null and its first char != '\0' or strlen >
0) and return JNI_TRUE only for a non-empty key.
In `@sdk/runanywhere-commons/src/backends/sarvam/rac_backend_sarvam_register.cpp`:
- Around line 106-127: The service factory sarvam_stt_create currently ignores
request->identifier when creating the backend (calls
rac_stt_sarvam_create(nullptr,...)) which causes the backend to always use the
default model while only service->model_id reflects the request; change this by
parsing request->identifier into a rac_stt_sarvam_config_t (or map supported
identifiers to a config/enum), pass that config pointer to rac_stt_sarvam_create
instead of nullptr, and on invalid/unsupported identifiers return nullptr (after
cleaning up) rather than silently accepting them; use rac_stt_sarvam_destroy on
error paths and keep writing the chosen model string into service->model_id.
In `@sdk/runanywhere-commons/src/jni/runanywhere_commons_jni.cpp`:
- Around line 4623-4697: The code races reading
g_http_executor_obj/g_http_executor_method in jni_http_executor because
racHttpSetExecutor updates them under g_http_executor_mutex; fix by taking a
stable snapshot of those two symbols under std::lock_guard<std::mutex>
lock(g_http_executor_mutex) (e.g., jobject local_executor = g_http_executor_obj;
jmethodID local_method = g_http_executor_method), check
local_executor/local_method for null and handle the error path, then use
local_executor/local_method for env->CallVoidMethod and later logic (do not hold
the mutex while calling Java). Ensure you still remove the pending callback
using g_pending_http_mutex as before and keep exception and cleanup handling
unchanged.
In `@sdk/runanywhere-kotlin/README.md`:
- Around line 344-351: Update the README section describing the cascade (the
paragraph around "confidence < 0.5" and the "How it works" steps) to explicitly
state that the current confidence signal is a mocked/random placeholder (not a
real model confidence) and that the 0.5 threshold is illustrative; mention that
routing metadata (which backend was used, whether it was a fallback, and
confidence scores) will include this mocked score until a real confidence metric
replaces it. Also add a short note indicating where to change this behavior in
code (i.e., the component that generates the confidence score) so maintainers
know to replace the placeholder with a real model-derived confidence in Whisper
before relying on cloud fallback behavior.
In `@sdk/runanywhere-kotlin/scripts/build-kotlin.sh`:
- Around line 430-436: The Sarvam native library copy block silently skips when
librac_backend_sarvam.so is missing; update the block in build-kotlin.sh (the
if/elif that uses COMMONS_DIST, COMMONS_BUILD, MAIN_JNILIBS_DIR and currently
calls log_info) to emit a clear warning via log_warn or log_error when neither
source exists, and when a strict mode flag (e.g., a new FAIL_ON_MISSING_SARVAM
env var) is set, exit non‑zero to fail-fast; ensure the message includes which
paths were checked (COMMONS_DIST/jni/${ABI} and
COMMONS_BUILD/${ABI}/src/backends/sarvam) so maintainers can diagnose missing
artifacts.
In
`@sdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/backends/stt/SarvamSTTBackend.kt`:
- Around line 55-63: The code currently checks CppBridgeSTT.getLoadedModelId()
for the substring "sarvam", which can falsely treat different Sarvam variants as
the desired model; change the logic in SarvamSTTBackend to compare the full
model id returned by CppBridgeSTT.getLoadedModelId() against the exact expected
id "sarvam:saarika:v2.5" (use equality, not contains) before skipping load, and
ensure any metadata or reporting (the same area that sets the model id/name
after load) uses the actual loaded id from CppBridgeSTT.getLoadedModelId() when
present so the backend reports the true model rather than always
"sarvam:saarika:v2.5".
In
`@sdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/cloud/sarvam/SarvamBridge.kt`:
- Around line 15-28: The JNI external methods on SarvamBridge (e.g.,
nativeHasApiKey) must be made private and accessed only via public wrapper
methods that call ensureLoaded() first; update SarvamBridge by changing each
external declaration to private (or private external) and add a corresponding
public wrapper (e.g., hasApiKey()) that calls ensureLoaded() then delegates to
the private native method, doing this for every JNI entrypoint referenced
(nativeHasApiKey and the other external symbols in SarvamBridge) so first-time
calls won't throw UnsatisfiedLinkError.
In
`@sdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/foundation/bridge/CppBridge.kt`:
- Around line 155-157: CppBridge currently registers the HTTP executor via
CppBridgeHTTP.register() but never unregisters it on teardown; update the
CppBridge.shutdown() implementation to call the corresponding teardown API
(e.g., CppBridgeHTTP.unregister() or CppBridgeHTTP.shutdown()) so the HTTP
callbacks/executor are cleaned up, and ensure the call is placed alongside other
shutdown steps in CppBridge.shutdown() to symmetrically undo the registration
performed where CppBridgeHTTP.register() is invoked.
In
`@sdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/foundation/bridge/extensions/CppBridgeHTTP.kt`:
- Around line 269-356: executeBinaryHttpRequest in CppBridgeHTTP is bypassing
the class-level hooks: call the registered requestInterceptor (to allow
header/auth mutation) before building/sending the HttpURLConnection and call
requestListener (or requestListener.onResponse/onError) after receiving response
or on exceptions; specifically invoke the same hook points used by register()’s
JVM path (use requestInterceptor to update the mutable headers map and apply
those headers to connection, and invoke requestListener with the request
metadata and response status/body or errorMessage before calling
RunAnywhereBridge.racHttpExecutorComplete), and ensure errors
(SocketTimeoutException, SSLException, generic Exception) also trigger the
listener error callback so native traffic follows the same
interception/telemetry flow as other backends.
- Around line 305-334: The current debug logs in CppBridgeHTTP (calls to
CppBridgePlatformAdapter.logCallback using TAG) print the full headers map
(variable headers) and the full non-2xx response body (responseStr); update
those two log sites to redact sensitive values and limit payloads: replace full
header logging with a sanitized headers map that masks values for keys like
Authorization, Api-Key, X-Api-Key, Cookie, and any header containing "token" or
"secret", and when logging non-success bodies only log a sanitized/trimmed
summary (e.g., JSON keys without values, hashed/masked values, or first N chars
+ length) rather than the complete responseStr; ensure this sanitization runs
immediately before the two logCallback calls so TAG, statusCode and timing
remain logged but no raw secrets are emitted.
In
`@sdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/native/bridge/RunAnywhereBridge.kt`:
- Around line 462-466: The external JNI function racHttpSetExecutor currently
takes a non-null Any but the native C++ expects nullptr to unregister; change
the Kotlin declaration to accept a nullable callback (Any?) so callers can pass
null to unregister and allow CppBridgeHTTP.unregister() to perform cleanup;
update any callers to pass null when unregistering and ensure the JNI
binding/signature is kept in sync with the native implementation and the
external declaration racHttpSetExecutor(callback: Any?) so the TODO in
CppBridgeHTTP.kt can be implemented.
In
`@sdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/public/extensions/RunAnywhere`+STT.jvmAndroid.kt:
- Around line 79-99: The code is overwriting the backend-reported confidence
with a random value (mockConfidence) causing nondeterministic fallbacks; replace
usage of mockConfidence with the native confidence returned by
WhisperSTTBackend.transcribe() (the confidence on result/STTOutput) when
constructing resultWithRouting, logging, and the confidence cascade decision
(use result.confidence or a safe default if nullable), and remove the
Random.nextFloat() assignment so cascading and exposed STTOutput.confidence
reflect the backend's actual score.
In
`@sdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/public/PlatformBridge.kt`:
- Line 62: The global HybridRouterRegistry initialized by
HybridRouterRegistry.initialize() needs a matching teardown: update
shutdownPlatformBridge() to clear/release the registry state by adding and
calling a reset/cleanup method on HybridRouterRegistry (e.g.,
HybridRouterRegistry.reset() or shutdown()) so backend registrations and global
routing state are cleared before process reuse; implement the reset method
inside HybridRouterRegistry to remove registrations and any listeners/threads in
a thread-safe manner and invoke it from shutdownPlatformBridge().
In
`@sdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/routing/HybridRouterRegistry.kt`:
- Around line 40-57: Move the isInitialized flip to after successful
registration and perform the whole init sequence under a lock: in initialize()
acquire a synchronization block (e.g., synchronized(this) or a dedicated mutex),
re-check isInitialized inside the lock, then create the backends list and call
router.register(backend) / populate sttBackends using backend.descriptors();
only after all backends are registered successfully set isInitialized = true.
Also wrap the registration loop in a try/catch so that on exception you rollback
any partial state (unregister any already-registered backends and clear
sttBackends) and rethrow the exception so the singleton is not left marked
initialized.
In
`@sdk/runanywhere-kotlin/src/jvmMain/kotlin/com/runanywhere/sdk/routing/NetworkAvailability.jvm.kt`:
- Line 9: The isNetworkAvailable() function is incorrectly hardcoded to always
return true; replace this stub with a real JVM network check (mirror the iOS
implementation behavior) by inspecting system network interfaces: iterate
NetworkInterface.getNetworkInterfaces(), return true if any interface isUp() and
!isLoopback() and hasInetAddresses(), otherwise return false; catch and
log/ignore exceptions and return false on error so cloud eligibility won't be
assumed when offline.
---
Outside diff comments:
In `@sdk/runanywhere-commons/scripts/build-android.sh`:
- Around line 347-360: In the CMake invocation block where cmake is run to
configure Android builds, change the RAG backend flag from -DRAC_BACKEND_RAG=ON
to -DRAC_BACKEND_RAG=OFF and add the platform-off flag -DRAC_BUILD_PLATFORM=OFF
to the argument list so RAG is disabled and platform-specific build components
are turned off; update the cmake command that contains -DRAC_BACKEND_RAG and the
surrounding flags (e.g., -DRAC_BUILD_BACKENDS, -DRAC_BUILD_JNI) to include
-DRAC_BUILD_PLATFORM=OFF and set -DRAC_BACKEND_RAG=OFF.
In
`@sdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/public/extensions/RunAnywhere`+ModelManagement.jvmAndroid.kt:
- Around line 94-103: The SARVAM framework mapping is only handled in the write
path causing reads to downgrade to UNKNOWN; update the read/path mappings to
include SARVAM too: in bridgeModelToPublic (and any framework-int parsing
function used when converting CppBridgeModelRegistry.Framework back to
InferenceFramework) add the branch mapping
CppBridgeModelRegistry.Framework.SARVAM <-> InferenceFramework.SARVAM so
modelInfo.framework and the registry enum round-trip without losing SARVAM.
---
Minor comments:
In `@docs/impl/hybrid-routing.md`:
- Around line 23-32: The fenced ASCII flow diagrams (e.g., the block starting
with "Record full audio" and the other similar diagrams further down) lack
language identifiers and trigger markdownlint; update each triple-backtick fence
to include a language tag such as ```text (or ```txt) so markdownlint stops
flagging them, and apply the same fix to the other two fenced blocks containing
the ASCII diagrams/reflow (the ones used for the Whisper->Sarvam routing
examples) to ensure consistency.
- Around line 177-179: The doc note about NetworkAvailability.kt is stale:
update the hybrid-routing.md section that references NetworkAvailability.kt to
reflect that network checks are implemented using Kotlin expect/actual rather
than Android reflection; remove the mention of Android reflection and add a
brief line indicating that platform-specific implementations live in
expect/actual declarations (e.g., the NetworkAvailability expect in common and
actual implementations on Android/iOS), and update any file list or path
comments to point to the expect/actual implementation locations instead of
implying reflection-based behavior.
In
`@examples/android/RunAnywhereAI/app/src/main/java/com/runanywhere/runanywhereai/presentation/stt/SpeechToTextViewModel.kt`:
- Around line 613-616: Reset the routing metadata fields routingBackendId,
routingBackendName, wasFallback, and primaryConfidence whenever you clear
transcription/metrics or begin a new session: update the places that clear
transcription/metrics (e.g., startCapture()/startRecording(),
clearResults()/clearTranscription(), and the error path such as
handleTranscriptionError()/onTranscriptionFailed()) to also set routingBackendId
and routingBackendName to null (or empty), wasFallback to false, and
primaryConfidence to a neutral value (e.g., 0.0), so stale backend/fallback
banners cannot persist across recordings or after failures.
In `@sdk/runanywhere-commons/src/backends/sarvam/rac_stt_sarvam.cpp`:
- Around line 132-136: The current rac_stt_sarvam_get_api_key() returns
key.c_str() after releasing the mutex which can be invalidated; change the API
to avoid returning a dangling pointer by either (A) implementing
rac_stt_sarvam_has_api_key() that acquires the
rac::sarvam::global_api_key_mutex(), checks !global_api_key().empty(), returns a
bool and update callers such as SarvamBridge.nativeHasApiKey() to use it, or (B)
make rac_stt_sarvam_get_api_key() return a heap-allocated copy (caller must
free) by locking the mutex, copying the string into a newly allocated char* and
returning that pointer and document the ownership, and ensure
rac_stt_sarvam_set_api_key() still protects writes with
rac::sarvam::global_api_key_mutex(); prefer option A for minimal churn.
In `@sdk/runanywhere-commons/src/backends/sarvam/README.md`:
- Around line 118-122: The fenced code block listing files (the triple-backtick
block containing "rac_stt_sarvam.h", "rac_stt_sarvam.cpp",
"rac_backend_sarvam_register.cpp") needs a language tag to satisfy markdown
linting; change the opening ``` to ```text or ```plaintext in the README.md so
the block is treated as plain text and static analysis warnings are resolved.
In
`@sdk/runanywhere-kotlin/src/androidInstrumentedTest/kotlin/com/runanywhere/sdk/routing/STTRoutingInstrumentedTest.kt`:
- Around line 19-20: Replace the inline Gradle instruction that currently shows
"./gradlew :connectedAndroidTest" in the file header comment with the
repository-standard SDK wrapper invocation (e.g. "./scripts/sdk.sh
:connectedAndroidTest"); update the comment line containing the exact string
"./gradlew :connectedAndroidTest" so contributors use the SDK wrapper script
instead of raw Gradle.
---
Nitpick comments:
In `@CLAUDE.md`:
- Line 330: Replace the non-navigable ellipsis in the pattern string
"'.../routing/'" with the concrete directory/package path used in the repository
for the runanywhere-kotlin routing module (i.e., the full commonMain routing
package path referenced elsewhere), so readers can click/navigate directly;
update the CLAUDE.md sentence that currently shows
"sdk/runanywhere-kotlin/src/commonMain/.../routing/" to use that full path
instead of the ellipsis.
In `@sdk/runanywhere-commons/scripts/build-android.sh`:
- Around line 589-593: The current copy for sarvam .so only checks
"${ABI_BUILD_DIR}/src/backends/sarvam/librac_backend_sarvam.so" and silently
skips if missing; update the block around that check to try a fallback path
(e.g. "${ABI_BUILD_DIR}/backends/sarvam/librac_backend_sarvam.so") and copy from
it if present, and if neither path exists emit a warning message consistent with
other backend copy logic (use ABI_BUILD_DIR, JNI_DIST_DIR, ABI and the filename
librac_backend_sarvam.so in the messages).
In `@sdk/runanywhere-commons/tests/test_stt_sarvam.cpp`:
- Around line 524-529: The test currently allocates 180s of PCM via
generate_pcm_sine(440.0f, 180.0f) which produces ~5.76MB and can slow CI; change
the duration to a minimal value just over the 2-minute limit (e.g., 121 seconds)
so that the rac_stt_sarvam_transcribe call still triggers the "audio too long"
error and the ASSERT_TRUE(rc != RAC_SUCCESS, ...) assertion remains valid;
update the call site where generate_pcm_sine is invoked and ensure pcm.size()
usage and the rac_stt_sarvam_transcribe invocation remain unchanged.
- Around line 289-296: The test uses an arbitrary cast (0x1) to rac_handle_t
when calling rac_stt_sarvam_transcribe, which can cause undefined behavior if
the implementation dereferences the handle; replace the bogus handle with a
benign valid handle value (e.g., construct a local rac_handle_t
fake_handle/zero-initialized struct or use an existing
RAC_INVALID_HANDLE/allocator helper) and pass &fake_handle to
rac_stt_sarvam_transcribe in the two null-parameter assertions so the test
exercises null-audio/null-result validation without risking handle dereference
UB.
In
`@sdk/runanywhere-kotlin/src/commonMain/kotlin/com/runanywhere/sdk/routing/HybridRouter.kt`:
- Around line 96-101: The code calls computeScore(it, context) twice per
candidate (once for sorting and again inside the debug log); change the
implementation in HybridRouter to compute and cache each candidate's score once
(e.g., map candidates to pairs of (candidate, score) or a small data class),
sort by the cached score, then produce the sorted candidate list and use the
cached score values in the logger.debug message (referencing computeScore,
capability, logger.debug and the sorted candidate list) so scores are not
recomputed.
In
`@sdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/cloud/sarvam/Sarvam.kt`:
- Around line 71-76: The conditional checking for -20 is dead/misleading; update
the Sarvam registration logic by removing the -20 branch and treating any
non-zero result from SarvamBridge.nativeRegister() as a failure: call
logger.error("Sarvam registration failed: $result") and return when result != 0.
Also remove or correct the comment that references -20 and mention that the JNI
wrapper normalizes the already-registered case (nativeRegister() returns 0 for
already-registered).
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 7ebea8a2-0180-4179-a0fb-2542c7c5d871
📒 Files selected for processing (57)
AGENTS.mdCLAUDE.mddocs/impl/hybrid-routing.mdexamples/android/RunAnywhereAI/app/src/main/java/com/runanywhere/runanywhereai/data/ModelList.ktexamples/android/RunAnywhereAI/app/src/main/java/com/runanywhere/runanywhereai/presentation/models/ModelSelectionViewModel.ktexamples/android/RunAnywhereAI/app/src/main/java/com/runanywhere/runanywhereai/presentation/navigation/MoreHubScreen.ktexamples/android/RunAnywhereAI/app/src/main/java/com/runanywhere/runanywhereai/presentation/stt/SpeechToTextScreen.ktexamples/android/RunAnywhereAI/app/src/main/java/com/runanywhere/runanywhereai/presentation/stt/SpeechToTextViewModel.ktsdk/runanywhere-commons/CMakeLists.txtsdk/runanywhere-commons/include/rac/backends/rac_stt_sarvam.hsdk/runanywhere-commons/include/rac/core/rac_error.hsdk/runanywhere-commons/include/rac/infrastructure/model_management/rac_model_types.hsdk/runanywhere-commons/include/rac/infrastructure/network/rac_http_client.hsdk/runanywhere-commons/scripts/build-android.shsdk/runanywhere-commons/src/backends/sarvam/CMakeLists.txtsdk/runanywhere-commons/src/backends/sarvam/README.mdsdk/runanywhere-commons/src/backends/sarvam/jni/rac_backend_sarvam_jni.cppsdk/runanywhere-commons/src/backends/sarvam/rac_backend_sarvam_register.cppsdk/runanywhere-commons/src/backends/sarvam/rac_stt_sarvam.cppsdk/runanywhere-commons/src/backends/sarvam/rac_stt_sarvam.hsdk/runanywhere-commons/src/core/capabilities/lifecycle_manager.cppsdk/runanywhere-commons/src/core/rac_error.cppsdk/runanywhere-commons/src/jni/runanywhere_commons_jni.cppsdk/runanywhere-commons/tests/CMakeLists.txtsdk/runanywhere-commons/tests/test_stt_sarvam.cppsdk/runanywhere-kotlin/README.mdsdk/runanywhere-kotlin/build.gradle.ktssdk/runanywhere-kotlin/scripts/build-kotlin.shsdk/runanywhere-kotlin/src/androidInstrumentedTest/kotlin/com/runanywhere/sdk/routing/STTRoutingInstrumentedTest.ktsdk/runanywhere-kotlin/src/androidMain/kotlin/com/runanywhere/sdk/routing/NetworkAvailability.android.ktsdk/runanywhere-kotlin/src/commonMain/kotlin/com/runanywhere/sdk/core/types/ComponentTypes.ktsdk/runanywhere-kotlin/src/commonMain/kotlin/com/runanywhere/sdk/public/extensions/STT/STTTypes.ktsdk/runanywhere-kotlin/src/commonMain/kotlin/com/runanywhere/sdk/routing/BackendDescriptor.ktsdk/runanywhere-kotlin/src/commonMain/kotlin/com/runanywhere/sdk/routing/HybridRouter.ktsdk/runanywhere-kotlin/src/commonMain/kotlin/com/runanywhere/sdk/routing/NetworkAvailability.ktsdk/runanywhere-kotlin/src/commonMain/kotlin/com/runanywhere/sdk/routing/RoutableBackend.ktsdk/runanywhere-kotlin/src/commonMain/kotlin/com/runanywhere/sdk/routing/RoutingCondition.ktsdk/runanywhere-kotlin/src/commonMain/kotlin/com/runanywhere/sdk/routing/RoutingContext.ktsdk/runanywhere-kotlin/src/commonMain/kotlin/com/runanywhere/sdk/routing/RoutingPolicy.ktsdk/runanywhere-kotlin/src/commonMain/kotlin/com/runanywhere/sdk/routing/RoutingResult.ktsdk/runanywhere-kotlin/src/commonMain/kotlin/com/runanywhere/sdk/routing/STTBackend.ktsdk/runanywhere-kotlin/src/commonTest/kotlin/com/runanywhere/sdk/SarvamSTTTest.ktsdk/runanywhere-kotlin/src/commonTest/kotlin/com/runanywhere/sdk/routing/HybridRouterTest.ktsdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/backends/stt/SarvamSTTBackend.ktsdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/backends/stt/WhisperSTTBackend.ktsdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/cloud/sarvam/Sarvam.ktsdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/cloud/sarvam/SarvamBridge.ktsdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/foundation/bridge/CppBridge.ktsdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/foundation/bridge/extensions/CppBridgeHTTP.ktsdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/foundation/bridge/extensions/CppBridgeModelRegistry.ktsdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/native/bridge/RunAnywhereBridge.ktsdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/public/PlatformBridge.ktsdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/public/extensions/RunAnywhere+ModelManagement.jvmAndroid.ktsdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/public/extensions/RunAnywhere+STT.jvmAndroid.ktsdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/routing/HybridRouterRegistry.ktsdk/runanywhere-kotlin/src/jvmMain/kotlin/com/runanywhere/sdk/routing/NetworkAvailability.jvm.ktsdk/runanywhere-kotlin/src/jvmTest/kotlin/com/runanywhere/sdk/SDKTest.kt
💤 Files with no reviewable changes (1)
- sdk/runanywhere-kotlin/src/jvmTest/kotlin/com/runanywhere/sdk/SDKTest.kt
...ples/android/RunAnywhereAI/app/src/main/java/com/runanywhere/runanywhereai/data/ModelList.kt
Show resolved
Hide resolved
...ereAI/app/src/main/java/com/runanywhere/runanywhereai/presentation/stt/SpeechToTextScreen.kt
Outdated
Show resolved
Hide resolved
...AI/app/src/main/java/com/runanywhere/runanywhereai/presentation/stt/SpeechToTextViewModel.kt
Show resolved
Hide resolved
sdk/runanywhere-commons/src/backends/sarvam/jni/rac_backend_sarvam_jni.cpp
Show resolved
Hide resolved
...here-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/native/bridge/RunAnywhereBridge.kt
Show resolved
Hide resolved
...rc/jvmAndroidMain/kotlin/com/runanywhere/sdk/public/extensions/RunAnywhere+STT.jvmAndroid.kt
Show resolved
Hide resolved
sdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/public/PlatformBridge.kt
Show resolved
Hide resolved
...nywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/routing/HybridRouterRegistry.kt
Show resolved
Hide resolved
...runanywhere-kotlin/src/jvmMain/kotlin/com/runanywhere/sdk/routing/NetworkAvailability.jvm.kt
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
♻️ Duplicate comments (1)
examples/android/RunAnywhereAI/app/src/main/java/com/runanywhere/runanywhereai/data/ModelList.kt (1)
193-205:⚠️ Potential issue | 🟠 MajorUse a config-provided Sarvam key and skip registration when it’s missing.
Line 194 still registers Sarvam with a literal placeholder. That means the cloud fallback path is either dead on startup or will fail later at first use while the backend appears registered. Read the key from a non-committed source (for example a
BuildConfigfield/local config) and only runSarvam.register(...)plusRunAnywhere.registerModel(...)when a non-blank key is available.Suggested shape
// Cloud backends try { - Sarvam.register(apiKey = "YOUR_SARVAM_API_KEY") + val sarvamApiKey = getSarvamApiKeyOrNull() + if (sarvamApiKey.isNullOrBlank()) { + Timber.i("Sarvam disabled: missing API key") + return + } + + Sarvam.register(apiKey = sarvamApiKey) // Register Sarvam model in C++ registry (not in UI model lists) RunAnywhere.registerModel( id = "sarvam:saarika:v2.5",Based on learnings: Refer to
docs/impl/hybrid-routing.mdfor the SDK's hybrid routing system architecture, confidence cascade mechanism, API key setup, and instructions for adding new STT/LLM/TTS providers.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@examples/android/RunAnywhereAI/app/src/main/java/com/runanywhere/runanywhereai/data/ModelList.kt` around lines 193 - 205, Replace the hardcoded Sarvam API key by reading it from a non-committed config (e.g., BuildConfig.SARVAM_API_KEY or a local config provider) and guard registration: only call Sarvam.register(...) and RunAnywhere.registerModel(...) for the "sarvam:saarika:v2.5" model when the retrieved key is non-blank; if the key is missing/blank, skip both Sarvam.register and RunAnywhere.registerModel so the cloud STT backend is not falsely advertised as available.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Duplicate comments:
In
`@examples/android/RunAnywhereAI/app/src/main/java/com/runanywhere/runanywhereai/data/ModelList.kt`:
- Around line 193-205: Replace the hardcoded Sarvam API key by reading it from a
non-committed config (e.g., BuildConfig.SARVAM_API_KEY or a local config
provider) and guard registration: only call Sarvam.register(...) and
RunAnywhere.registerModel(...) for the "sarvam:saarika:v2.5" model when the
retrieved key is non-blank; if the key is missing/blank, skip both
Sarvam.register and RunAnywhere.registerModel so the cloud STT backend is not
falsely advertised as available.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 30d4d530-4205-4960-838d-1fefc3212d02
📒 Files selected for processing (1)
examples/android/RunAnywhereAI/app/src/main/java/com/runanywhere/runanywhereai/data/ModelList.kt
- Fix HybridRouterRegistry race condition (synchronized + flag after init) - Redact API keys and auth headers from HTTP logs - Truncate transcription text in logs (PII) - Pass detectedLanguage in WhisperSTTBackend - Fix routing label: check backend name not just wasFallback - Guard SarvamBridge JNI methods with ensureLoaded() - Validate API key not empty in JNI layer - Fix HTTP executor race in JNI (mutex-guarded reads) - Add shutdown path for HybridRouterRegistry + CppBridge HTTP cleanup - Exact model ID prefix match in SarvamSTTBackend - Warn on missing Sarvam .so in build script - JVM network check uses socket probe instead of always-true
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In
`@sdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/routing/HybridRouterRegistry.kt`:
- Around line 26-27: The router keeps descriptors after teardown causing
resolveSTT() to return stale candidates; in shutdown() clear the router’s
registered descriptors as well as sttBackends by either calling a router API to
remove/unregister all descriptors (e.g., router.clear/removeAll/unregisterAll)
or make router mutable (change private val router to var) and reassign router =
HybridRouter() in shutdown(); update initialize(), shutdown(), and any
registration paths that add descriptors so the router and sttBackends stay in
sync.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: d7b6fbf0-adc4-4057-b848-42def390fcd9
📒 Files selected for processing (13)
examples/android/RunAnywhereAI/app/src/main/java/com/runanywhere/runanywhereai/presentation/stt/SpeechToTextScreen.ktexamples/android/RunAnywhereAI/app/src/main/java/com/runanywhere/runanywhereai/presentation/stt/SpeechToTextViewModel.ktsdk/runanywhere-commons/src/backends/sarvam/jni/rac_backend_sarvam_jni.cppsdk/runanywhere-commons/src/jni/runanywhere_commons_jni.cppsdk/runanywhere-kotlin/scripts/build-kotlin.shsdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/backends/stt/SarvamSTTBackend.ktsdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/backends/stt/WhisperSTTBackend.ktsdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/cloud/sarvam/SarvamBridge.ktsdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/foundation/bridge/CppBridge.ktsdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/foundation/bridge/extensions/CppBridgeHTTP.ktsdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/public/PlatformBridge.ktsdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/routing/HybridRouterRegistry.ktsdk/runanywhere-kotlin/src/jvmMain/kotlin/com/runanywhere/sdk/routing/NetworkAvailability.jvm.kt
🚧 Files skipped from review as they are similar to previous changes (8)
- sdk/runanywhere-kotlin/src/jvmMain/kotlin/com/runanywhere/sdk/routing/NetworkAvailability.jvm.kt
- sdk/runanywhere-kotlin/scripts/build-kotlin.sh
- sdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/foundation/bridge/CppBridge.kt
- examples/android/RunAnywhereAI/app/src/main/java/com/runanywhere/runanywhereai/presentation/stt/SpeechToTextScreen.kt
- examples/android/RunAnywhereAI/app/src/main/java/com/runanywhere/runanywhereai/presentation/stt/SpeechToTextViewModel.kt
- sdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/backends/stt/SarvamSTTBackend.kt
- sdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/public/PlatformBridge.kt
- sdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/foundation/bridge/extensions/CppBridgeHTTP.kt
...nywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/routing/HybridRouterRegistry.kt
Show resolved
Hide resolved
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a routing layer that picks between Whisper (local) and Sarvam (cloud) for STT. Local runs first. If confidence is low, same audio goes to cloud. Whisper gets reloaded after so next request stays local.
commonMain— no C++ changes for routing itselfRandom.nextFloat()for confidence — swap with real inference confidence laterAlso includes the Sarvam C++ backend + JNI bridge from the previous work (language fix for the 400 error).
What changed
sdk/runanywhere-kotlin/src/commonMain/.../routing/— 9 new files: router, conditions, policies, backend interfacessdk/runanywhere-kotlin/src/jvmAndroidMain/.../backends/stt/— Whisper and Sarvam backend wrapperssdk/runanywhere-kotlin/src/jvmAndroidMain/.../RunAnywhere+STT.jvmAndroid.kt— cascade logic, model restoresdk/runanywhere-kotlin/src/androidMain/+jvmMain/— expect/actual for network checkexamples/android/— routing info in STT UI, removed Cloud STT screen, Sarvam hidden from model pickersdk/runanywhere-commons/— Sarvam C++ backend, JNI bridge, language field fixdocs/impl/hybrid-routing.md— how it works, how to add providers, API key setupHow to add a new provider
Implement
STTBackend, register inHybridRouterRegistry.initialize(). Thats it.Test plan
./gradlew testDebugUnitTest --tests "com.runanywhere.sdk.routing.*")./gradlew connectedDebugAndroidTest)Summary by CodeRabbit
New Features
Documentation
Tests
Greptile Summary
This PR adds a hybrid STT routing layer that runs Whisper locally first and cascades to Sarvam AI cloud when confidence is low, along with the Sarvam C++ backend, JNI bridge, and routing UI in the Android demo. The routing engine itself (
HybridRouter,RoutingCondition,BackendDescriptor) is well-designed, but there are two blockers that must be fixed before merge:ModelList.ktand will be committed to git history — the key should be revoked immediately and sourced from a secrets store or local config.Random.nextFloat()as a placeholder, which means ~50% of all Whisper transcriptions will silently fall back to the paid cloud backend, incurring real API costs and latency with no quality benefit.Confidence Score: 2/5
Not safe to merge — a live API key is hardcoded in source and the random confidence mock will silently bill users on ~50% of requests.
Two blockers: a P0 credential leak that needs immediate revocation and removal, and a P1 logic bug where Random.nextFloat() drives real cloud API calls at production scale. The routing architecture itself is sound, but these must be resolved before the PR ships.
examples/android/RunAnywhereAI/app/src/main/java/com/runanywhere/runanywhereai/data/ModelList.kt (hardcoded key) and sdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/public/extensions/RunAnywhere+STT.jvmAndroid.kt (random confidence mock).
Security Review
ModelList.ktline 193): A Sarvam AI secret key is committed directly in source. The key is now in git history and accessible to anyone with repository read access. Revoke immediately.rac_stt_sarvam.cpp):global_api_key()is a static string protected only by a mutex. If the process is inspected via debugger or memory dump the key is readable in plaintext — acceptable for an SDK but worth documenting.Important Files Changed
Sequence Diagram
sequenceDiagram participant App participant transcribeWithOptions participant HybridRouterRegistry participant HybridRouter participant WhisperSTTBackend participant SarvamSTTBackend participant CppBridgeSTT App->>transcribeWithOptions: audioData + STTOptions transcribeWithOptions->>HybridRouterRegistry: resolveSTT(context) HybridRouterRegistry->>HybridRouter: resolve(STT, context) HybridRouter-->>transcribeWithOptions: [whisper-local, sarvam-cloud] transcribeWithOptions->>WhisperSTTBackend: transcribe(audio, options) WhisperSTTBackend->>CppBridgeSTT: transcribe(audio, config) CppBridgeSTT-->>WhisperSTTBackend: result WhisperSTTBackend-->>transcribeWithOptions: STTOutput note over transcribeWithOptions: mockConfidence = Random.nextFloat() threshold=0.5 alt confidence >= 0.5 (~50% of requests) transcribeWithOptions-->>App: STTOutput (local) else confidence < 0.5 (~50% of requests) transcribeWithOptions->>SarvamSTTBackend: transcribe(audio, options) SarvamSTTBackend->>CppBridgeSTT: loadModel(sarvam) note over SarvamSTTBackend: HTTP to Sarvam API SarvamSTTBackend-->>transcribeWithOptions: STTOutput transcribeWithOptions->>CppBridgeSTT: restoreLocalModel transcribeWithOptions-->>App: STTOutput (wasFallback=true) endPrompt To Fix All With AI
Reviews (1): Last reviewed commit: "Add routing tests and implementation doc..." | Re-trigger Greptile