Symptom. First AI cleanup after launch, or after any gap >5 min, takes ~10s before text returns. Subsequent cleanups are fast until the next 5 min gap.
Cause. IDLE_TIMEOUT_MS = 5 * 60 * 1000 unloads the llama server on idle. The next cleanup re-reads the full GGUF from disk. With Gemma 4 E4B (5.1GB, Q4_K_M) that read is most of the 10s. Load time, not inference time.
Why it's the common path. Dictation is bursty: short utterance, then minutes of reading or editing. Most bursts land past the 5 min window, so cold is the norm, not the exception.
Gap. #1078 refers to a Parakeet + Whisper pre-warm. STT appears pre-warmed; the reasoning model isn't.
Proposal. Start the reasoning model load on record-hotkey-down rather than after transcription completes, so it overlaps the recording. No change to the idle timeout; unload behavior unchanged.
Happy to take the PR if that's the right hook.
Env. Windows 11, OpenWhispr 1.7.5, Ryzen 9 5900X, RTX 3070 8GB, 32GB RAM. Parakeet Unified EN (STT) + Gemma 4 E4B Q4_K_M (reasoning), Vulkan llama server.
Symptom. First AI cleanup after launch, or after any gap >5 min, takes ~10s before text returns. Subsequent cleanups are fast until the next 5 min gap.
Cause.
IDLE_TIMEOUT_MS = 5 * 60 * 1000unloads the llama server on idle. The next cleanup re-reads the full GGUF from disk. With Gemma 4 E4B (5.1GB, Q4_K_M) that read is most of the 10s. Load time, not inference time.Why it's the common path. Dictation is bursty: short utterance, then minutes of reading or editing. Most bursts land past the 5 min window, so cold is the norm, not the exception.
Gap. #1078 refers to a Parakeet + Whisper pre-warm. STT appears pre-warmed; the reasoning model isn't.
Proposal. Start the reasoning model load on record-hotkey-down rather than after transcription completes, so it overlaps the recording. No change to the idle timeout; unload behavior unchanged.
Happy to take the PR if that's the right hook.
Env. Windows 11, OpenWhispr 1.7.5, Ryzen 9 5900X, RTX 3070 8GB, 32GB RAM. Parakeet Unified EN (STT) + Gemma 4 E4B Q4_K_M (reasoning), Vulkan llama server.