Add experiment driven inline model feedback survey

[Logan Ramos (lramos15)]

Adds an inline feedback survey that can be attached to chat responses through an experiment treatment, so a survey can be authored, changed, or retired without shipping code.

When a survey applies to a response, a combined thumbs up and down control replaces the usual helpful and unhelpful actions in the footer. Pressing it opens a short multi step survey underneath, and pressing it again closes it.

How a survey is configured

Everything comes from a versioned JSON payload delivered as the chatModelFeedbackSurvey treatment:

{
  "version": 1,
  "id": "auto-routing-2026-08",
  "match": { "selectedModels": ["auto"] },
  "prompt": {
    "cooldownDays": 7,
    "maxPerSession": 1,
    "chance": { "initial": 0.05, "increment": 0.05, "max": 0.5 },
    "triggers": { "modelSwitchedAway": { "enabled": true, "bypassCooldown": true } }
  },
  "steps": [
    { "kind": "choice", "id": "routing", "title": "Did Auto choose the right model for the job?",
      "options": [ { "id": "yes", "label": "Yes" }, { "id": "too-heavy", "label": "No - too heavy" } ] },
    { "kind": "text", "id": "comments", "title": "Anything else you'd like to share? (optional)",
      "placeholder": "Optional feedback", "maxLength": 1000 }
  ]
}

A malformed payload is rejected whole rather than in part, since dropping one bad step would quietly change what the experiment measures. An omitted prompt block gives a manual only survey, so a config that forgets to describe its pacing under prompts rather than nags.

Model identifiers are not qualified the same way across harnesses. The language model service uses <vendor>/<group>/<id> while agent host sessions use <sessionType>:<id>, so a selector is matched against each segment as well as the whole id. That lets auto match both copilot/auto and agent-host-copilotcli:auto.

Manual and automatic surfacing are separate

Manual activation is never rate limited, because the pacing rules exist only to pace surveys the user did not ask for. Those rules cover a weekly cooldown, a per session cap, a probability that ramps with each response that passes without prompting, and a trigger for switching off the surveyed model.

Only the newest response offers a survey, and only one is open at a time, so history never fills with stale controls. A survey the user is part way through is never pulled away by a newer response or an automatic prompt.

Telemetry

Answers are reported as each step is taken, so surveys the user abandons still produce data. Events reach GitHub restricted telemetry through a command the Copilot extension registers; a command is used rather than a data channel because invoking it activates the extension, so results produced before activation are not dropped. Free text only ever reaches the restricted sink, and nothing is sent when telemetry.feedback.enabled is off or telemetry is disabled.

Core already has a sender for the same table in agentHostRestrictedTelemetry.ts using the same ingestion key, but it is node layer inside the agent host process with no channel to the renderer. Exposing it would let this forwarder and the existing GithubTelemetryForwardingContrib both go away; that is worth a follow up.

Accessibility

The panel is a card with a listbox of options, matching the ask question tool. Arrow keys, Home, End, Enter, and Space move and pick, and the widget stops those keys from also reaching the chat list. Escape dismisses. Focus moves into the survey only when the user asked for it, and returns to the feedback control when the panel closes. The active option carries aria-selected so screen readers announce what will be submitted.

Notes for review

• isLast was added to IChatResponseViewModel. The getter already existed on the single implementer.
• ToolBar.focus now forwards an optional index, which ActionBar.focus already accepted.

Testing

46 tests covering config validation and matching, prompt pacing, the manual and automatic paths, state lifecycle across the virtualized list, keyboard and ARIA behaviour, and the telemetry contract. The full chat suite passes at 3242.

[Copilot]

Pull request overview

Adds experiment-configured inline model feedback surveys to chat responses, including automatic prompting, accessible UI, and restricted telemetry forwarding.

Changes:

• Adds validated survey configuration, matching, pacing, and lifecycle management.
• Adds the inline survey widget and response-footer action.
• Forwards survey results through the Copilot extension with comprehensive tests.

Show a summary per file

File	Description
chatModelFeedbackSurveyConfig.test.ts	Tests payload parsing and matching.
chatListWidget.test.ts	Stubs the survey service.
chatListRenderer.test.ts	Updates renderer test dependencies.
mockChatModelFeedbackSurveyService.ts	Adds a no-op test service.
chatModelFeedbackSurveyWidget.test.ts	Tests keyboard, ARIA, and rendering behavior.
chatModelFeedbackSurveyService.test.ts	Tests lifecycle, pacing, and telemetry.
chatModelFeedbackSurveyPromptContribution.test.ts	Tests model-switch detection.
chatModelFeedbackSurveyActions.test.ts	Tests footer-control focus restoration.
chatViewModel.ts	Exposes response isLast.
chatModelFeedbackSurveyTelemetry.ts	Defines the telemetry wire contract.
chatModelFeedbackSurveyConfig.ts	Parses, validates, and matches survey payloads.
chatContextKeys.ts	Adds survey visibility and open-state keys.
chatListRenderer.ts	Renders surveys beneath response footers.
chatModelFeedbackSurvey.css	Styles the survey card and footer control.
chatModelFeedbackSurveyWidget.ts	Implements survey interaction and accessibility behavior.
chatModelFeedbackSurveyService.ts	Manages survey state, pacing, and reporting.
chatModelFeedbackSurveyPromptContribution.ts	Observes model-picker changes.
chat.shared.contribution.ts	Registers survey services and contributions.
chatTitleActions.ts	Replaces standard voting actions when applicable.
chatModelFeedbackSurveyActions.ts	Adds the combined feedback action.
toolbar.ts	Supports focusing a toolbar item by index.
chatModelFeedbackSurveyForwardingContrib.ts	Forwards restricted survey telemetry.
contributions.ts	Registers telemetry forwarding.
package.json	Adds command-based extension activation.

Review details

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

• Files reviewed: 24/24 changed files
• Comments generated: 6
• Review effort level: Balanced