Summary
Add higher-accuracy speech-to-text backends behind the existing core.Transcriber interface (internal/core/interfaces.go), so users can opt into much better recognition than the default.
Context
- STT today is native-first:
SpeechAnalyzer/SpeechTranscriber in the Swift helper (macos-helper/Sources/Helper/Capture/LiveSpeechPipeline.swift), adapted to Go via internal/stt.
DESIGN.md/CLAUDE.md already anticipate whisper.cpp as an optional post-v1 fallback behind the same interface.
Scope
- Integrate whisper.cpp (and leave room for other models) as an alternative transcriber, selectable via
internal/config like the distiller/answerer providers.
- Keep the hard invariant: audio never crosses the IPC boundary; any whisper backend that needs audio must run inside the helper (
macos-helper/), emitting only Utterance text to the core.
- Model management: download/select model size, document accuracy vs latency/resource tradeoffs.
- Benchmark accuracy against the native transcriber on a fixed fixture (
test/fixtures/speech.wav) so the win is measurable.
Open questions
- whisper.cpp in-helper (Swift bridging) vs a separate on-device process; both must keep audio local.
- Streaming vs chunked transcription and its effect on segmentation latency.
Related: multilingual support (separate issue) largely rides on model choice.
Summary
Add higher-accuracy speech-to-text backends behind the existing
core.Transcriberinterface (internal/core/interfaces.go), so users can opt into much better recognition than the default.Context
SpeechAnalyzer/SpeechTranscriberin the Swift helper (macos-helper/Sources/Helper/Capture/LiveSpeechPipeline.swift), adapted to Go viainternal/stt.DESIGN.md/CLAUDE.mdalready anticipate whisper.cpp as an optional post-v1 fallback behind the same interface.Scope
internal/configlike the distiller/answerer providers.macos-helper/), emitting onlyUtterancetext to the core.test/fixtures/speech.wav) so the win is measurable.Open questions
Related: multilingual support (separate issue) largely rides on model choice.