Fix silent wake: never suppress the first utterance after wake - #12
Fix silent wake: never suppress the first utterance after wake#12jayzuccarelli wants to merge 2 commits into
Conversation
The model sometimes classified the user's first post-wake utterance as background and called wait_for_user, stranding the session silent until hygiene timeout (live incident 2026-07-27 01:22 UTC). The wake word is proof that first speech is addressed. - instructions: turn 1 is always addressed, never wait_for_user it - server: if wait_for_user still fires on the first committed turn, answer instead of suppressing (instruction alone fixed only 1/5) - harness: new first_turn_addressed scenario, turn-1 twin of tv_line_after_answer (same TV line: answer on turn 1, ignore on turn 2) Bench (5 reps, fresh sessions): first_turn_addressed 0/5 on main -> 5/5 with fix; background_rejection 5/5 both; tv_line_after_answer 4/5 both (pre-known flake, no override fired). make check GREEN 2/2. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
# Conflicts: # broker/realtime_broker/agent.py
|
Deployed to the live container 2026-08-12 as image |
f53bb69 to
e0b8b80
Compare
1 participant
Problem
Sometimes Mycroft wakes, hears the question, and answers with nothing: the model classifies the user's first post-wake utterance as background speech and calls
wait_for_user, which intentionally suppresses the response. The session then strands silent until broker hygiene closes it (~16s of spinning LED). Live incident: 2026-07-27 01:22 UTC, the onlywait_for_usercall in 72h of logs, fired on a real question.The wake word is proof the first speech is addressed: nobody says "hey Mycroft" for the TV.
Fix (broker only)
wait_for_userit.wait_for_useron the first committed turn (_TurnHygiene.is_first_turn(), counter resets per device connect), the broker answers instead of suppressing, with a warning log. Needed: with the instruction alone the model still misfired 4/5.first_turn_addressedscenario, deliberately the same TV line astv_line_after_answer: the pair encodes the tradeoff (answer on turn 1, ignore on turn 2).Bench (5 reps each, fresh sessions, isolated brokers on 8766/8767, real OpenAI + live HA MCP)
No regression; the single tv_line failure is symmetric and pre-known flaky, and the override fired 0 times during both counter-metric soaks (correctly scoped to turn 1). Overridden turn-1 answers cost ~880ms p50 first-audio (extra tool round trip) vs 0-420ms direct. Turn-2 rejection with an override in context: 4/4 silent.
make checkGREEN 2/2.Known tradeoff (accepted)
On a TV false wake (~0.567/h at current sensitivity), the first TV line now gets answered aloud instead of ignored silently. Previously those sessions died silent after 16s. New exposure this creates: command-like TV speech on a false wake ("turn it up") can now drive real HA tool calls — the bench observed spurious attempts (pause show, find TV, volume) on overridden TV-line answers; all failed, no state changed, but the surface exists. Bounded by turn hygiene in production.
Not fixed here
VAD misses of the first utterance (root cause 2 of the silent-wake report) and the firmware no-answer timeout. Separate levers.
Deploy is gated separately; live container untouched.
🤖 Generated with Claude Code