feat: add wake phrase voice assistant preview

[calebeden]

Additional instructions

MUST: Keep Allow edits from maintainers enabled for this PR so maintainers can help update the branch when needed.

What Problem This Solves

Windows users do not have an always-on, wake-phrase voice experience comparable to the macOS app. Existing speech input, gateway chat, and speech output work independently, but users must manually start each interaction.

Why This Change Was Made

This adds a default-off Assistant preview that reuses the existing local Whisper STT, energy VAD, canonical gateway chat path, and configured TTS provider. V0 deliberately supports one flow only: say the wake phrase and request in one utterance, receive one spoken reply, then return to listening. It adds no wake-word model, protocol command, persistent conversation mode, barge-in, or overlay.

User Impact

Users can configure a custom one-to-three-word wake phrase on Voice & Audio after configuring speech input and Companion Voice. When enabled, a matching utterance sends only the request after the wake phrase, capitalizes its first character, speaks the correlated reply once, and resumes listening.

While Assistant owns the microphone for wake listening, native Chat visibly explains why single-message voice input is unavailable. Turning Assistant off releases capture, removes that status when Chat is shown again, and restores normal voice transcription.

Evidence

• Full repository build passed on Windows ARM64 with .NET SDK 10.0.301.
• Shared tests: 3,399 passed, 32 skipped.
• Tray tests: 2,086 passed, 0 skipped.
• Focused wake-gate and coordinator tests: 24 passed.
• Automated coverage includes wake matching, settings persistence, readiness, exclusive microphone ownership, response correlation and deduplication, repeated turns, cancellation, timeout, provider replacement, localization, accessibility feedback, and preserved-chat state refresh.

Change Type

• Bug fix
• Feature
• Refactor
• Docs or instructions
• Tests or validation
• Security hardening
• Chore or infrastructure

Scope

• Tray or WinUI UX
• Windows node capability
• Local MCP or winnode
• Gateway, connection, or pairing
• Setup or onboarding
• Permissions, privacy, or security
• Tests, CI, or docs

Validation

• ./build.ps1: passed
• dotnet test ./tests/OpenClaw.Shared.Tests/OpenClaw.Shared.Tests.csproj --no-restore: 3,399 passed, 32 skipped
• dotnet test ./tests/OpenClaw.Tray.Tests/OpenClaw.Tray.Tests.csproj --no-restore: 2,086 passed, 0 skipped
• dotnet test ./tests/OpenClaw.Tray.Tests/OpenClaw.Tray.Tests.csproj --no-restore --filter "FullyQualifiedName~VoiceWakeGateTests|FullyQualifiedName~VoiceAssistantCoordinatorTests": 24 passed

Real Behavior Proof

• Environment tested: Windows 11 ARM64, .NET SDK 10.0.301, Windows SDK 10.0.26100.0
• PR head or commit tested: 19e4b163da3ad9d0b5a2368a65da3046c6387ec2
• Exact steps or command run: Enabled the Assistant preview with configured STT and TTS, spoke the configured wake phrase and request in one utterance, and observed the gateway response and spoken playback.
• Video recording:

OpenClawAssistantV0.mp4

• Observed result: The recording demonstrates a successful configured wake-to-request-to-spoken-reply turn on the current PR head.
• Screenshot or artifact links verified? (Yes/No/N/A): Yes, the recording resolves as an MP4
• Not verified or blocked: The recording proves the end-to-end wake flow. Microphone contention and recovery are covered by automated tests but are not separately demonstrated in this recording.

Security Impact

• New permissions or capabilities? (Yes/No): No
• Secrets or tokens handling changed? (Yes/No): No
• New or changed network calls? (Yes/No): No
• Command or tool execution surface changed? (Yes/No): No
• Data access scope changed? (Yes/No): Yes
• If any answer is Yes, explain the risk and mitigation: When explicitly enabled, the microphone remains active and local Whisper transcribes silence-bounded speech bursts to detect the configured phrase. The feature is off by default, requires existing STT and TTS capabilities, does not log audio or transcript content, and sends only the extracted request through the existing configured gateway path.

Compatibility and Migration

• Backward compatible? (Yes/No): Yes
• Config or environment changes? (Yes/No): Yes, two optional Assistant settings are persisted with safe defaults
• Migration needed? (Yes/No): No
• If yes, list the exact upgrade steps: N/A

Review Conversations

• I replied to or resolved every bot review conversation addressed by this PR.
• I left unresolved only conversations that still need maintainer judgment.

[clawsweeper]

Codex review: found issues before merge. Reviewed August 1, 2026, 1:57 PM ET / 17:57 UTC.

ClawSweeper review

What this changes

Adds a default-off Windows tray preview that listens locally for a configured wake phrase, sends one extracted request through gateway chat, and speaks the resulting reply.

Merge readiness

⛔ Blocked by patch quality or review findings - 11 items remain

Keep this PR open. Its happy-path video proves the preview can complete one turn, but current code still leaves the assistant infrastructure alive when the setting is turned off and lacks a demonstrated gateway-run binding for reply ownership, matching the blocking collaborator review.

Priority: P1
Reviewed head: 19e4b163da3ad9d0b5a2368a65da3046c6387ec2
Owner decision: Required. See Decision needed.

Review scores

Measure	Result	What it means
Overall readiness	🧂 unranked krab (1/6)	The PR has strong happy-path media proof, but P1 run-ownership and teardown defects make the current patch unsafe to merge.
Proof confidence	🦞 diamond lobster (5/6) ✨ media proof bonus	Sufficient (recording): The linked MP4 and PR description credibly show one successful wake-to-request-to-spoken-reply turn on the current head, although they do not prove the concurrent cancellation and recovery cases.
Patch quality	🧂 unranked krab (1/6)	Security review found an item that needs attention.

Verification

Check	Result	Evidence
Real behavior	Verified	Sufficient (recording): The linked MP4 and PR description credibly show one successful wake-to-request-to-spoken-reply turn on the current head, although they do not prove the concurrent cancellation and recovery cases.
Evidence reviewed	5 items	Current main does not implement the preview: Current main contains bounded STT and TTS settings and chat voice input, but no VoiceAssistant coordinator, wake-one-shot setting, or wake phrase route. The requested feature is therefore neither merged nor present in the current release. Turning the feature off retains the coordinator: The settings path calls ReconcileAsync whenever a coordinator already exists, rather than disposing it. That retains its event subscriptions and lifecycle objects after the user selects Off, as the collaborator review identified. Reply identity does not carry the gateway run: The proposed receipt includes SendRunId, but stored response identity contains only session, gateway message metadata, and response text. The response-observation path therefore has no exact run identity to compare before a response is claimed.
Findings	2 actionable findings	[P1] Dispose assistant infrastructure when the preview turns off [P1] Bind observed replies to the receipt's gateway run
Security	Needs attention	Scope spoken replies to the initiating gateway run: The reply identity path does not preserve the receipt's gateway run id, leaving same-thread finals vulnerable to incorrect assistant ownership and spoken disclosure.

How this fits together

The tray voice stack owns Windows microphone capture and local speech transcription, while the chat provider sends user requests to the configured gateway and receives final replies. This PR inserts a coordinator between those paths so wake-triggered speech can produce one correlated spoken response before listening resumes.

flowchart LR
  A[Windows microphone] --> B[Local speech transcription]
  B --> C[Wake phrase gate]
  C --> D[Assistant coordinator]
  D --> E[Gateway chat request]
  E --> F[Correlated final reply]
  F --> G[Configured text to speech]
  D --> H[Voice and Audio settings]

Loading

Decision needed

Question	Recommendation
Should OpenClaw accept a core, default-off always-listening microphone preview after the run-isolation fixes, or should this capability remain out of the core tray product for now?	Sponsor a narrowed preview after repair: Keep the default-off feature direction, require exact-run correlation and full disposal on Off, then obtain fresh race and microphone-recovery proof.

Why: The patch introduces a new persistent microphone workflow and user-visible assistant mode. Correctness fixes are necessary, but they cannot establish whether this privacy-sensitive feature belongs in the core product.

Before merge

• Dispose assistant infrastructure when the preview turns off (P1) - When a coordinator already exists, this path only reconciles it. Selecting Off must dispose the coordinator, input adapter, and chat-turn client so their subscriptions and microphone-related lifecycle machinery cannot survive after the feature is disabled.
• Bind observed replies to the receipt's gateway run (P1) - The receipt records SendRunId, but response identity does not retain or compare it. A final from another active run in the same session can therefore be observed under the assistant's local message identity and later spoken. Carry and verify the exact gateway run before claiming a reply.
• Resolve security concern: Scope spoken replies to the initiating gateway run - The reply identity path does not preserve the receipt's gateway run id, leaving same-thread finals vulnerable to incorrect assistant ownership and spoken disclosure.
• Resolve merge risk (P1) - A same-thread final can be associated with the assistant without proving it belongs to the receipt's gateway run, which can make the app speak unrelated content.
• Resolve merge risk (P2) - Timeout, disable, recovery, and Stop races may cancel or suppress a different active gateway run unless cancellation is guarded by the exact active receipt.
• Resolve merge risk (P1) - The preview keeps a microphone capture pipeline active while enabled, so product approval must explicitly cover the persistent-capture privacy behavior and recovery semantics.
• Resolve merge risk (P1) - The current head has a failing revocation-recovery E2E check, so the PR's listed local happy-path validation is not sufficient merge readiness evidence.
• Improve patch quality - Preserve and compare the exact gateway run id for response claiming and cancellation.
• Improve patch quality - Dispose all assistant objects when settings change to Off, then add regression coverage for re-enable and late-response races.
• Improve patch quality - Provide redacted current-head runtime proof for Stop, disable, timeout, and recovery while another chat run is active.

Findings

• [P1] Dispose assistant infrastructure when the preview turns off — src/OpenClaw.Tray.WinUI/App.xaml.cs:4823-4825
• [P1] Bind observed replies to the receipt's gateway run — src/OpenClaw.Tray.WinUI/Services/VoiceAssistant/VoiceAssistantChatTurnClient.cs:92-96
• [medium] Scope spoken replies to the initiating gateway run — src/OpenClaw.Tray.WinUI/Services/VoiceAssistant/VoiceAssistantChatTurnClient.cs:92

Agent review details

Security

Needs attention: The default-off microphone feature has a concrete response-ownership boundary problem: it can speak content not established as the reply to the initiating voice request.

Review metrics

Metric	Value	Why it matters
Patch breadth	53 files, 3,134 added, 82 removed	The feature crosses microphone capture, settings, gateway chat correlation, WinUI surfaces, localization, and tests, increasing the cost of an unsafe lifecycle or correlation assumption.
Focused defects	2 source-confirmed P1 findings	Both affect ownership of user-visible speech or the lifetime of microphone-related infrastructure.

Merge-risk options

Maintainer options:

1. Repair exact-run ownership before merge (recommended)
   Carry the gateway run identity through response observation and cancellation, reject late or suppressed responses before claiming them, and dispose the coordinator when Off is selected.
2. Pause pending product sponsorship
   Do not merge the persistent microphone preview until a maintainer explicitly accepts the core-product privacy and lifecycle contract.

Technical review

Best possible solution:

Keep the preview default-off, but repair it around a single run-scoped receipt: only the exact active run may claim a reply or be aborted, suppression must precede claiming, and switching Off must dispose all coordinator subscriptions and capture resources before the feature can be reconsidered.

Do we have a high-confidence way to reproduce the issue?

No for current main, because this preview is PR-only. The proposed head is source-reproducible for the disable-lifetime defect, and the collaborator review provides a concrete concurrent-run scenario for the response and abort defects.

Is this the best way to solve the issue?

No. The current approach needs exact gateway-run ownership and deterministic teardown before it can safely compose microphone capture, gateway chat, and TTS playback.

Full review comments:

• [P1] Dispose assistant infrastructure when the preview turns off — src/OpenClaw.Tray.WinUI/App.xaml.cs:4823-4825
  When a coordinator already exists, this path only reconciles it. Selecting Off must dispose the coordinator, input adapter, and chat-turn client so their subscriptions and microphone-related lifecycle machinery cannot survive after the feature is disabled.
  Confidence: 0.98
• [P1] Bind observed replies to the receipt's gateway run — src/OpenClaw.Tray.WinUI/Services/VoiceAssistant/VoiceAssistantChatTurnClient.cs:92-96
  The receipt records SendRunId, but response identity does not retain or compare it. A final from another active run in the same session can therefore be observed under the assistant's local message identity and later spoken. Carry and verify the exact gateway run before claiming a reply.
  Confidence: 0.9

Overall correctness: patch is incorrect
Overall confidence: 0.93

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 8ac00c5f4977.

Labels

Label justifications:

• P1: An enabled assistant can incorrectly claim or cancel an active chat turn, affecting real gateway conversations and spoken output.
• merge-risk: 🚨 security-boundary: The new persistent microphone path can speak a response not proven to belong to the initiating request.
• merge-risk: 🚨 availability: Disable, timeout, and recovery races can leave capture or gateway-turn handling in an unsafe state.
• rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🦞 diamond lobster and patch quality is 🧂 unranked krab.
• status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action. Sufficient (recording): The linked MP4 and PR description credibly show one successful wake-to-request-to-spoken-reply turn on the current head, although they do not prove the concurrent cancellation and recovery cases.
• proof: sufficient: Contributor real behavior proof is sufficient. The linked MP4 and PR description credibly show one successful wake-to-request-to-spoken-reply turn on the current head, although they do not prove the concurrent cancellation and recovery cases.
• proof: 🎥 video: Contributor real behavior proof includes video or recording evidence. The linked MP4 and PR description credibly show one successful wake-to-request-to-spoken-reply turn on the current head, although they do not prove the concurrent cancellation and recovery cases.

Evidence

Security concerns:

• [medium] Scope spoken replies to the initiating gateway run — src/OpenClaw.Tray.WinUI/Services/VoiceAssistant/VoiceAssistantChatTurnClient.cs:92
  The reply identity path does not preserve the receipt's gateway run id, leaving same-thread finals vulnerable to incorrect assistant ownership and spoken disclosure.
  Confidence: 0.9

What I checked:

• Current main does not implement the preview: Current main contains bounded STT and TTS settings and chat voice input, but no VoiceAssistant coordinator, wake-one-shot setting, or wake phrase route. The requested feature is therefore neither merged nor present in the current release. (src/OpenClaw.Tray.WinUI/Pages/VoiceSettingsPage.xaml:159, 8ac00c5f4977)
• Turning the feature off retains the coordinator: The settings path calls ReconcileAsync whenever a coordinator already exists, rather than disposing it. That retains its event subscriptions and lifecycle objects after the user selects Off, as the collaborator review identified. (src/OpenClaw.Tray.WinUI/App.xaml.cs:4825, 19e4b163da3a)
• Reply identity does not carry the gateway run: The proposed receipt includes SendRunId, but stored response identity contains only session, gateway message metadata, and response text. The response-observation path therefore has no exact run identity to compare before a response is claimed. (src/OpenClaw.Tray.WinUI/Services/VoiceAssistant/VoiceAssistantChatTurnClient.cs:92, 19e4b163da3a)
• Blocking review remains on the current head: A collaborator review records unresolved run-correlation, cancellation, stop-suppression, and disable-disposal defects. The reviewed head remains 19e4b16, so no subsequent contributor commit addresses that review. (19e4b163da3a)
• Release check: The PR head is not an ancestor of the v0.6.12 release and is not merged into current main, so the new preview has no release provenance. (ef9935161a25)

Likely related people:

• shanselman: Provided collaborator review identified the remaining run-correlation, cancellation, stop-suppression, and disposal boundaries. (role: blocking design reviewer; confidence: high; files: src/OpenClaw.Tray.WinUI/App.xaml.cs, src/OpenClaw.Tray.WinUI/Services/VoiceAssistant/VoiceAssistantCoordinator.cs)
• Barbara Kudiess: Recent main history extracted Settings-page persistence into the settings-store/view-model boundary that this PR extends. (role: recent settings and tray architecture contributor; confidence: high; commits: d2638ddbef57; files: src/OpenClaw.Tray.WinUI/App.xaml.cs, src/OpenClaw.Tray.WinUI/Presentation/SettingsStore.cs)
• Caleb Eden: Prior merged main history includes native Windows chat lifecycle work, and the proposed coordinator relies on that chat-provider path. (role: recent chat-area contributor; confidence: medium; commits: 2fb1da26c175, 282e583e7257; files: src/OpenClaw.Tray.WinUI/Chat/OpenClawChatDataProvider.cs, src/OpenClaw.Tray.WinUI/Chat/OpenClawChatCoordinator.cs)

Rating scale

Score	Internal tier	Crab rank	Meaning
6/6	S	🦀 challenger crab	Exceptional readiness
5/6	A	🦞 diamond lobster	Very strong readiness
4/6	B	🐚 platinum hermit	Good normal PR; ordinary maintainer review
3/6	C	🦐 gold shrimp	Useful, but confidence is limited
2/6	D	🦪 silver shellfish	Proof or implementation needs work
1/6	F	🧂 unranked krab	Not merge-ready
N/A	NA	🌊 off-meta tidepool	Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

• ClawSweeper keeps one durable marker-backed review comment per issue or PR.
• Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
• A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
• PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
• Maintainers can also comment @clawsweeper review to request a fresh review only.
• Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
• Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
• Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

History

Review history (13 earlier review cycles; latest 8 shown)

• reviewed 2026-07-31T19:16:11.035Z sha 19e4b16 :: needs real behavior proof before merge. :: none
• reviewed 2026-07-31T20:34:48.299Z sha 19e4b16 :: needs real behavior proof before merge. :: none
• reviewed 2026-07-31T22:39:06.540Z sha 19e4b16 :: needs real behavior proof before merge. :: [P1] Dispose the assistant coordinator when Assistant is turned off
• reviewed 2026-08-01T01:54:23.329Z sha 19e4b16 :: needs changes before merge. :: [P1] Dispose the assistant coordinator when Assistant is turned off
• reviewed 2026-08-01T10:19:37.806Z sha 19e4b16 :: needs changes before merge. :: none
• reviewed 2026-08-01T12:21:46.492Z sha 19e4b16 :: needs changes before merge. :: [P1] Check suppression before claiming the assistant response | [P1] Dispose assistant infrastructure when the preview is turned off
• reviewed 2026-08-01T13:52:36.524Z sha 19e4b16 :: needs changes before merge. :: [P1] Check suppression before claiming the assistant response | [P1] Dispose assistant infrastructure when the preview is turned off
• reviewed 2026-08-01T16:41:16.587Z sha 19e4b16 :: needs changes before merge. :: [P1] Check suppression before claiming assistant responses | [P1] Dispose the assistant coordinator when the preview is turned off | [P1] Match assistant finals against the exact gateway run | [P1] Abort only the still-active receipt run

[shanselman]

Adversarial maintainer review found blocking run-correlation defects, so this is not ready to land yet:

• The assistant can claim and speak an unrelated final message from the same thread because correlation is thread-scoped rather than gateway-run-scoped.
• Timeout, TTS failure, disable, or recovery can call the thread-scoped abort path and terminate another active run, or queue an abort against the next run.
• Readiness changes can cancel the assistant's own in-flight turn, and the UI Stop path is not coordinated before a late response is claimed/spoken.
• Turning the feature off reconciles to Off but leaves the coordinator, subscriptions, and capture machinery alive.

Please bind the turn to the gateway run id, make cancellation no-op unless that exact run is still active, move abort suppression ahead of response claiming, and dispose the coordinator when disabled. The current revocation-recovery check is also red. Current merge confidence is 45%.