Skip to content

agentHost: improve Agent Merge CI diagnostics - #335501

Merged
Benjamin Christopher Simmonds (benibenj) merged 2 commits into
mainfrom
agents/agent-merge-ci-diagnostics-improvements
Sep 10, 2026
Merged

agentHost: improve Agent Merge CI diagnostics#335501
Benjamin Christopher Simmonds (benibenj) merged 2 commits into
mainfrom
agents/agent-merge-ci-diagnostics-improvements

Conversation

@benibenj

Copy link
Copy Markdown
Contributor

Summary

Agent Merge previously returned only the first 100,000 characters of job logs whose downloads were already limited to a 2 MiB prefix. This could hide the actual failing-test summary with no supported way to read further.

  • Make readAgentMergeCI summary-first, with failed checks, annotations, jobs/steps, and bounded failure excerpts under a 48,000-byte response budget.
  • Add job selection, line-numbered tail/range reads, literal search with context, and actionable continuation cursors to the same authorized tool.
  • Capture workflow logs up to 16 MiB with the existing 30-second download deadline. Preserve whole-capture redaction, bound mask-processing work, and report incomplete prefixes explicitly instead of presenting them as true tails.
  • Cache redacted evidence with repository/account, PR/head, turn, workflow-attempt, and job identity; revalidate authorization on every read. Keep the cache bounded to eight logs / 32 Mi UTF-16 characters, expiring five minutes after their last summary or on eviction.
  • Queue concurrent reads cancellably and paginate summaries within cache capacity, so assembling a page cannot evict its own advertised evidence. Retain transport cancellation, cleanup, and credential/redirect safeguards.

No GitHub permission expansion, repair retry scheduling / blocked-outcome changes, or archive-nudge/onboarding changes.

Validation

  • npm run gulp compile-client — passed with zero errors.
  • npm run typecheck-client — passed.
  • Repository hygiene — all 12 changed files passed.
  • 115 focused unit tests passed across Agent Merge diagnostics, server tools, reruns, restrictions, GitHub transport, and PR mutation service.
  • Coverage includes failure summaries beyond both old cutoffs, exact-cap EOF versus incomplete downloads, redaction/chunk boundaries, stable range/search pagination, stale/unauthorized evidence, expiry/eviction, queued cancellation/disposal, six failed jobs, and cache-aware paging of four 12 MiB logs and twenty small logs.

The aggregate npm run compile-client also builds unrelated extensions and encountered a missing extension-side esbuild dependency in this isolated worktree. The source-only compile task above and the final compiled unit tests passed. No remote CI logs were retrieved for this change.

Replace prefix-only CI dumps with bounded summaries and authorized, cached tail/range/search reads. Preserve true EOF completeness, redaction, cancellation, and workflow-attempt identity while keeping responses and retained evidence bounded.

Serialize concurrent reads and paginate summaries within cache capacity so follow-up evidence remains actionable without changing repair scheduling.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings September 10, 2026 16:23

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Paginated searches repeatedly rescan prior lines, making late matches in newline-dense logs potentially exceed the tool deadline.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review tier: Balanced
Findings: 1 Medium severity

New issues introduced by this change (1)
Severity Finding
Medium severity src/​vs/​platform/​agentHost/​node/​agentMergeCIEvidence.ts — Each search continuation calls linesInRange with a later first, but that generator still walks…
What changed in this PR

Improves Agent Merge CI diagnostics with bounded summaries, navigable cached evidence, and safer workflow-log handling.

Changes:

  • Adds summary, tail, range, search, and continuation support.
  • Expands log capture to 16 MiB with redaction and authorization safeguards.
  • Adds comprehensive transport, lifecycle, caching, and diagnostics tests.
File Description
src/​vs/​platform/​github/​test/​node/​pullRequestMutationService.test.ts Tests attempts, job metadata, limits, and redaction.
src/​vs/​platform/​github/​test/​node/​githubTransport.test.ts Tests bounded streaming, cancellation, and redirects.
src/​vs/​platform/​github/​common/​pullRequestMutationService.ts Expands and redacts workflow-log capture.
src/​vs/​platform/​github/​common/​githubTransport.ts Hardens bounded downloads and cleanup.
src/​vs/​platform/​github/​common/​githubPullRequestMutationService.ts Extends workflow diagnostics contracts.
src/​vs/​platform/​agentHost/​test/​node/​agentMergeServerTools.test.ts Tests tool descriptions and request parsing.
src/​vs/​platform/​agentHost/​test/​node/​agentMergeRerun.test.ts Updates rerun integration expectations and disposal.
src/​vs/​platform/​agentHost/​test/​node/​agentMergeCITools.test.ts Covers diagnostics, caching, paging, and authorization.
src/​vs/​platform/​agentHost/​node/​shared/​agentMergeServerTools.ts Defines diagnostic modes and validation.
src/​vs/​platform/​agentHost/​node/​agentServiceComposition.ts Owns the disposable Agent Merge tools instance.
src/​vs/​platform/​agentHost/​node/​agentMergeTools.ts Implements summary-first authorized CI reads.
src/​vs/​platform/​agentHost/​node/​agentMergeCIEvidence.ts Implements evidence caching and navigation.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/vs/platform/agentHost/node/agentMergeCIEvidence.ts Outdated
Cache sparse line-start offsets so search continuations, context excerpts, ranges, and tails do not repeatedly scan logs from the beginning. Cover 500,000-line pagination, newline-dense 16 MiB logs, checkpoint boundaries, and empty evidence.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@benibenj
Benjamin Christopher Simmonds (benibenj) marked this pull request as ready for review September 10, 2026 17:49
@benibenj
Benjamin Christopher Simmonds (benibenj) merged commit 0b2d79f into main Sep 10, 2026
33 checks passed
@benibenj
Benjamin Christopher Simmonds (benibenj) deleted the agents/agent-merge-ci-diagnostics-improvements branch September 10, 2026 20:26
@vs-code-engineering vs-code-engineering Bot added this to the 1.138.0 milestone Sep 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants