Add Gemini 3.5 Transcribe as a BYOK speech-to-text provider - #7138
Merged
Conversation
Live captions use gemini-3.5-transcribe-live over the Live API, and post-recording refine maps that same key and live model id onto gemini-3.5-transcribe generateContent for speaker labels and word timestamps. Cloud Speech host matching is narrowed so Gemini's generativelanguage endpoint is not treated as google_cloud. Co-authored-by: John Jeong <ComputelessComputer@users.noreply.github.com>
✅ Deploy Preview for anarlog canceled.
|
|
|
ComputelessComputer
marked this pull request as ready for review
August 28, 2026 07:17
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 9fcdd6d. Configure here.
Nested ternaries after the new google_generative_ai branch were two spaces short, which failed fmt / fmt. Co-authored-by: John Jeong <ComputelessComputer@users.noreply.github.com>
Ordinary inputTranscription events are committed captions, not an explicit finalize. Only set from_finalize when Gemini marks the transcript finished after audioStreamEnd. Co-authored-by: John Jeong <ComputelessComputer@users.noreply.github.com>
Keep the Gemini adapter landed in #7137 and reapply this branch's live from_finalize mapping so only the audioStreamEnd flush is marked as a finalize. Co-authored-by: John Jeong <ComputelessComputer@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Problem: Gemini 3.5 Transcribe launched with a live streaming model and a batch model for pre-recorded audio, but Anarlog had no way to use either. Google Cloud Speech-to-Text is a different API, and host matching on
googleapis.comwould have treated Gemini's endpoint as Cloud Speech.Fix: Add Google Gemini as a bring-your-own-key STT provider. Live captions use
gemini-3.5-transcribe-liveover the Live API. After recording, the same API key maps that live model ontogemini-3.5-transcribefor speaker labels and word timestamps. Desktop talks to Google directly; this is not routed through the Anarlog STT proxy.Verification
cargo test -p owhisper-client --lib -- google_generativecargo test -p owhisper-client --lib -- adapter::testscargo test -p listener-core --libcargo test -p listener2-core --libcargo test -p transcribe-proxy --libpnpm -F desktop exec vitest run src/settings/ai/stt/shared.test.ts src/settings/ai/stt/selection.test.ts src/stt/capabilities.test.ts src/stt/useRunBatch.test.tsNot run against a real Gemini API key. Live sessions do not resume after the ~10-minute Live API cap, and batch uses inline
generateContent(about 20 MB / 15 min) rather than the Files API.