automations: fix: limit New badge to returning Agents users - #334898
Merged
Ulugbek Abdullaev (ulugbekna) merged 3 commits intoSep 7, 2026
Merged
Conversation
Copilot started reviewing on behalf of
Ulugbek Abdullaev (ulugbekna)
September 7, 2026 10:58
View session
Contributor
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
The aggregate discovery contract needs direct tests, and its documentation currently overstates unsupported-host readiness.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review tier: Balanced
Findings: 2
New issues introduced by this change (2)
| Severity | Finding |
|---|---|
 |
src/vs/sessions/AUTOMATIONS.md — This sentence is too broad: an unsupported host forwards its provider-scoped legacy store's… |
|
src/vs/sessions/contrib/automations/browser/providerAutomationService.ts — The aggregate precedence and reactivity introduced here are not exercised by… |
What changed in this PR
Restricts the Automations New badge to returning Agents-window users after authoritative Automation discovery.
Changes:
- Tracks prior Agents-window usage without double-counting telemetry.
- Adds provider discovery states and conservative badge eligibility.
- Expands unit tests, fixtures, and architecture documentation.
| File | Description |
|---|---|
src/vs/workbench/test/browser/componentFixtures/sessions/sessionsList.fixture.ts |
Updates badge fixtures for new dependencies. |
src/vs/workbench/contrib/chat/common/automations/automationService.ts |
Defines the discovery-state contract. |
src/vs/sessions/sessions.common.main.ts |
Loads the window-usage service. |
src/vs/sessions/services/sessions/test/browser/sessionsWindowUsageService.test.ts |
Tests usage counting. |
src/vs/sessions/services/sessions/common/sessionsProvider.ts |
Exposes optional provider discovery state. |
src/vs/sessions/services/sessions/browser/sessionsWindowUsageService.ts |
Records prior window usage. |
src/vs/sessions/contrib/sessions/test/browser/sessionsTelemetry.contribution.test.ts |
Updates telemetry test dependencies. |
src/vs/sessions/contrib/sessions/test/browser/sessionsListTestUtils.ts |
Stubs window-usage state. |
src/vs/sessions/contrib/sessions/test/browser/sessionsList.test.ts |
Updates accessible-label coverage. |
src/vs/sessions/contrib/sessions/test/browser/sessionsLifecycleTracker.test.ts |
Adapts lifecycle tracker tests. |
src/vs/sessions/contrib/sessions/test/browser/automationsNewBadge.test.ts |
Covers badge eligibility and suppression. |
src/vs/sessions/contrib/sessions/browser/sessionsTelemetry.contribution.ts |
Passes the captured launch count. |
src/vs/sessions/contrib/sessions/browser/sessionsLifecycleTracker.ts |
Removes duplicate launch counting. |
src/vs/sessions/contrib/sessions/browser/automationsNewBadge.ts |
Implements conservative badge lifecycle. |
src/vs/sessions/contrib/providers/agentHost/test/browser/localAgentHostSessionsProvider.test.ts |
Tests host reconnection discovery. |
src/vs/sessions/contrib/providers/agentHost/test/browser/agentHostAutomationStore.test.ts |
Expands discovery-state tests. |
src/vs/sessions/contrib/providers/agentHost/browser/reconnectableAgentHostAutomationStore.ts |
Maps connection state to discovery. |
src/vs/sessions/contrib/providers/agentHost/browser/localAgentHostSessionsProvider.ts |
Clears Automation connections on exit. |
src/vs/sessions/contrib/providers/agentHost/browser/agentHostAutomationStore.ts |
Tracks catalogue health and readiness. |
src/vs/sessions/contrib/automations/test/browser/providerAutomationService.test.ts |
Covers unreadable legacy discovery. |
src/vs/sessions/contrib/automations/test/browser/automationService.test.ts |
Tests ledger discovery states. |
src/vs/sessions/contrib/automations/browser/providerAutomationService.ts |
Aggregates provider discovery states. |
src/vs/sessions/contrib/automations/browser/automationService.ts |
Derives discovery from ledger readability. |
src/vs/sessions/AUTOMATIONS.md |
Documents initial discovery semantics. |
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Track prior Agents window opens with the existing machine-scoped counter. Gate badge presentation on provider discovery and Automation evidence while preserving the developer preview and experiment styles. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 7e75f732-f18f-42a7-b1e2-a034c03596eb
Include legacy ledger readability and live Agent Host catalogue health in badge eligibility. Propagate host disconnects, retain simple prior-window tracking, and add regression coverage without changing shared storage APIs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 7e75f732-f18f-42a7-b1e2-a034c03596eb
Exercise provider registration and live discovery precedence through the aggregate observable. Clarify that unsupported hosts are ready only when their legacy storage is readable. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 7e75f732-f18f-42a7-b1e2-a034c03596eb
Ulugbek Abdullaev (ulugbekna)
force-pushed
the
ulugbekna/agents/automation-badge-visibility-fix
branch
from
September 7, 2026 13:21
8667286 to
9a615e6
Compare
Ulugbek Abdullaev (ulugbekna)
marked this pull request as ready for review
September 7, 2026 15:34
Ulugbek Abdullaev (ulugbekna)
enabled auto-merge (squash)
September 7, 2026 15:34
Contributor
📬 CODENOTIFYThe following users are being notified based on files changed in this PR: Sandeep Somavarapu (@sandy081)Matched files:
Ladislau Szomoru (@lszomoru)Matched files:
|
Giuseppe Cianci (Giuspepe)
approved these changes
Sep 7, 2026
Ulugbek Abdullaev (ulugbekna)
deleted the
ulugbekna/agents/automation-badge-visibility-fix
branch
September 7, 2026 15:39
Ulugbek Abdullaev (ulugbekna)
added
the
~release-cherry-pick
vs-code-engineering
Bot
added
release-cherry-pick
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Restrict the Automations New badge to returning Agents-window users with no prior Automation discovery, and stop treating loading or failed catalogues as proof that Automations are unused.
Follow-up to #334091. The badge designs and renderer already exist; this changes eligibility, lifecycle handling, and related tests.
Rebased onto
70a1d06d93f6a90d3cae4bf84754b096d818ef40. The rebase reuses the canonicalcatalogueStateAPI introduced by #334836 instead of retaining a parallel discovery contract. The PR now changes 15 files rather than 24.Why
The previous badge sampled the seen bit, definitions, and runs once, synchronously. If all were empty, it resolved the style and displayed New.
That causes two problems:
Implementation
Returning-window detection without storage infrastructure changes
SessionsWindowUsageServicesnapshots whether the existing application-scoped, machine-targetedagentSessions.telemetry.summary.appLaunchCountwas nonzero before recording the current open.The first window therefore retains
hadPriorWindowOpen = falseeven after it stores count 1. Later windows, including users whose count was written by older builds, are recognized without relying on chat/session history.The usage service owns the increment;
SessionsLifecycleTrackerreceives the captured count throughSessionsTelemetryContributionrather than incrementing it a second time. The stored format and telemetry event schemas are unchanged, although the increment now occurs when the usage service is instantiated rather than specifically when the telemetry tracker is constructed.This uses ordinary application storage. No shared storage API, IPC, IndexedDB, or base/platform changes are included. Two simultaneous first opens can both read zero and remain unbadged; that conservative false negative is acceptable for a discovery cue.
Reuse upstream catalogue completeness
The badge reads the existing
IAutomationService.catalogueStateand accepts onlyready.loading,unavailable, anderrorall suppress it.Upstream already provides:
error > loading > unavailable > ready.The shared contract, aggregate and legacy implementations, Agent Host store implementations, provider interface, and architecture specification are unchanged from upstream. The former optional
initialDiscoveryStateAPI and its duplicated implementation are no longer part of this PR.The local provider still needs the lifecycle fix: clear its connection listeners and Automation connection on host exit, then use its existing start binding to reconnect. This prevents a stale ready catalogue from surviving local host shutdown.
Migrated definitions and execution remain Agent Host-owned. Legacy ledgers remain compatibility sources; no schema, scheduling, permission, or session-template change is introduced.
Conservative badge lifecycle
For an unseen returning user, the badge waits until
LifecyclePhase.Eventually, then evaluates the current aggregate once. It shows only when the catalogue is ready and definitions/runs are empty. It does not await a retrying migration promise.The startup phase is an opportunity for initial providers to populate, not a guarantee that every future provider has registered. The aggregate includes upstream's initial-provider-settlement guard; later data/readiness changes remain observable.
The existing hidden style setting, ExP styles, and accessible row labels are preserved. Style refreshes cannot bypass eligibility, and disposed states do not resume presentation after async work. The evidence observer is cleared on retirement.
Scope and tradeoffs
Validation
After rebasing, using fresh transpiled output:
npm run typecheck-clientpassed.git diff --checkpassed.The badge regressions cover all three non-ready catalogue states both at startup and after presentation, including recovery remaining quiet. The local-host regression starts with a completed catalogue and checks
ready -> unavailable -> readyacross exit/restart.The earlier review concerns remain covered: unsupported hosts never imply a ready catalogue, and upstream now directly tests provider registration, live aggregate updates, startup settlement, and the full precedence matrix in both provider orders. This PR also checks aggregate
errorfor a future-schema global legacy ledger.Earlier validation exercised all 15 badge variants across Dark, Light, and Dark High Contrast. Visual fixtures were not rerun after this rebase; no visual styles or screenshot baselines changed.
To rerun the focused tests:
npm run transpile-client ./scripts/test.sh \ --run src/vs/sessions/services/sessions/test/browser/sessionsWindowUsageService.test.ts \ --run src/vs/sessions/contrib/sessions/test/browser/sessionsLifecycleTracker.test.ts \ --run src/vs/sessions/contrib/sessions/test/browser/sessionsTelemetry.contribution.test.ts \ --run src/vs/sessions/contrib/sessions/test/browser/automationsNewBadge.test.ts \ --run src/vs/sessions/contrib/sessions/test/browser/sessionsList.test.ts \ --run src/vs/sessions/contrib/providers/agentHost/test/browser/agentHostAutomationStore.test.ts \ --run src/vs/sessions/contrib/automations/test/browser/providerAutomationService.test.ts \ --run src/vs/sessions/contrib/automations/test/browser/automationService.test.ts \ --run src/vs/sessions/contrib/providers/agentHost/test/browser/localAgentHostSessionsProvider.test.ts \ --grep '^(AutomationsNewBadgeState|SessionsWindowUsageService|SessionsLifecycleTracker|SessionsTelemetryContribution|Sessions - SessionsList|ProviderAutomationService|AutomationService|AgentHostAutomationStore|LocalAgentHostSessionsProvider Automation catalogue state follows)' \ --reporter dotFor visual iteration in a development Agents window, Developer: Reset Automations New Badge force-previews the current
sessions.automations.newBadgeStyle(accent,soft, oroutline). Opening Automations dismisses it. This previews presentation, not production returning-user eligibility.