agentHost: preserve BYOK state through tool continuations

[sgent-epic]

Summary

• retain provider response IDs only while tool calls are pending
• recover the Copilot SDK immediate tool-result replay by supplying previous_response_id and forwarding only the matching tool outputs
• scope pending state by authenticated SDK session, vendor, and model
• preserve explicit continuation IDs and clear recovered state after a terminal response

Why

The bundled Copilot SDK currently replays the full Responses history after a tool call and omits previous_response_id. Extension-provided stateful models then receive no stateful_marker and reject the continuation.

The integration test reproduces the SDK request shape directly. Before the proxy change, it failed because the second request replayed the message, reasoning, and tool call with no response ID. It now reaches the provider with the original response ID and only the tool output.

Fixes #329283

Testing

• npm run compile-client
• scripts\test.bat --run src\vs\platform\agentHost\test\node\byokLmProxyService.test.ts
• scripts\test-integration.bat --run src\vs\platform\agentHost\test\node\providerIntegration\copilotByokResponses.integrationTest.ts
• precommit hygiene
• git diff --check upstream/main...HEAD

[Copilot]

Pull request overview

Preserves BYOK provider state through Copilot SDK tool continuations, addressing #329283.

Changes:

• Tracks pending response IDs by session, vendor, and model.
• Recovers matching tool-output continuations without overriding explicit state.
• Adds unit and SDK integration coverage.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File	Description
byokLmProxyService.ts	Implements continuation recovery and scoped state tracking.
byokLmProxyService.test.ts	Tests recovery, scoping, explicit IDs, and cleanup.
copilotByokResponses.integrationTest.ts	Verifies SDK tool continuation behavior end-to-end.

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[sgent-epic]

Hi Vritant Bhardwaj (@vritant24), I hope you're doing well. Is there any way this could be prioritized? It's a blocker for wider piloting of agents in VS Code at Epic. I'm personally really liking the VS Code environment over other options, and I'd like to enable other folks to give it a try as well. Thanks for your time!

[Vritant Bhardwaj (vritant24)]

result.responseId is not always evidence that the downstream response is resumable.

• Explicit-ZDR Responses requests use store: false and deliberately remove previous_response_id.
• The Responses processor still emits the returned ID as stateful_marker.
• This proxy then caches that ID and replaces the SDK replay with only tool outputs.
• On the next request, the ZDR endpoint drops the injected marker, leaving orphaned tool outputs without the original history.

Suggested fix:

• Define stateful_marker as “resumable provider state”: do not emit it for explicit-ZDR/non-stored responses.
• With no marker, IByokLmChatResult.responseId remains absent and this proxy preserves the full stateless replay.
• Do not infer this from the local body.store: the SDK intentionally sends store: false to this facade even when the downstream provider is stateful.
• Add a ZDR Responses tool-continuation test verifying that no marker is cached and the full replay reaches the provider.

[sgent-epic]

Fixed in bf9576442cb. responseId is now explicitly an opt-in resumable-state contract. The component that knows whether a response is resumable owns whether to report it. The proxy does not infer this from store, and the regression verifies that absence preserves full replay without previous_response_id.

[Vritant Bhardwaj (vritant24)]

This retains only one continuation per (sessionId, vendor, modelId), but root, background, and subagent trajectories share the same session-scoped provider token.

A normal nested flow can therefore lose the parent state even without an HTTP race:

parent tool call   -> stores resp_parent
subagent response  -> overwrites/deletes the same entry
parent tool result -> recovery misses -> strict stateful provider rejects it

Suggested fix:

• Retain multiple pending responses and match trailing tool outputs against the exact call-ID/kind set.
• Recover only when exactly one candidate in the same session/vendor/model scope matches.
• Remove only the matched predecessor after a successful continuation; unrelated terminal responses must not clear other entries.
• Keep memory bounded with a flat global limit of 256 pending responses, not unbounded arrays per scope, and store only IDs/kinds rather than history.
• Add an interleaving test with parent and subagent continuations using the same session/vendor/model.

[sgent-epic]

Fixed in 6a805a3423f. The proxy now keeps a flat globally bounded set of candidates, resumes only a unique exact call-ID/kind match within the same scope, and removes only the matched predecessor after a successful bridge call. Tests cover parent/subagent interleaving, ambiguous matches, and the 256-entry bound.

[sgent-epic]

Hi Vritant Bhardwaj (@vritant24), please let me know if there are any changes required. Thanks!

[Vritant Bhardwaj (vritant24)]

Thanks for the update. The parent/subagent collision looks fixed now. I think two things remain:

• The ZDR path still emits every Responses ID as a statefulMarker, even though responses created with store: false cannot be resumed. The new test assumes responseId is already absent, but the real producer still supplies one. Could we suppress the marker for ZDR responses and test that path directly?

• Explicit continuations do not clear their cached entry. We only delete state when a continuation was implicitly recovered, so a successful request with previous_response_id leaves stale state behind. Could we remove the matching entry after that request succeeds, while retaining it on errors?

The latest rebase doesn’t appear to change either path.

[sgent-epic]

Thanks, Vritant Bhardwaj (@vritant24). Both are addressed in e2cee5c. ZDR responses no longer emit markers, with direct ZDR/stored coverage. Explicit continuations now remove matching cached state after success while retaining it on errors. Full compile and focused tests pass.

[Mohammad javad Dianat (dianatofficial)]

Verified the diff. Changes align with project standards.

[sgent-epic]

Hi Vritant Bhardwaj (@vritant24) can we run the test suites again? Thanks!

[Copilot]

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.

[sgent-epic]

Hi Vritant Bhardwaj (@vritant24), can you please help me with the next steps? Apologies for the nagging, but it's a blocker for us. TIA!

[sgent-epic]

Hi Vritant Bhardwaj (@vritant24) is there anything else I need to do? Can you please approve the workflows to run?