store: false still persists and resolves previous_response_id across WebSocket connections (breaks ZDR expectation)

[jtechapps]

Summary

On the WebSocket Responses endpoint, a response.create sent with store: false still causes the server to persist conversation state: the returned previous_response_id is later resolved on a brand-new WebSocket connection. Per the OpenAI Responses API spec, store: false (and the Zero Data Retention mode that forces it) means state is not persisted server-side and a subsequent previous_response_id lookup should not resolve — clients are expected to replay full history themselves. This divergence means data a ZDR client expected to be held only in memory (connection-local) is retained and retrievable across connections.

Expected behavior (per OpenAI spec)

With ZDR enabled at the org level, the store parameter for /v1/responses is always treated as false. Under store: false, response/reasoning items are not persisted, so chaining via previous_response_id fails with an error along the lines of "Items are not persisted when store is set to false. Try again with store set to true, or remove this item from your input." Stateless/ZDR clients must instead pass the full conversation history as input on each turn. See Data controls in the OpenAI platform and Conversation state or create new lane in the same connection.

Observed behavior (agentic-api)

A response.create with store: false returns a response.id. Sending that id as previous_response_id on a separate, freshly opened WebSocket connection resolves the prior context and the model recalls the earlier content — i.e. the state was persisted server-side and is retrievable across connections despite store: false.

Environment

• Endpoint: vLLM agentic-api WebSocket Responses endpoint (/v1/responses), reached via kubectl port-forward svc/agentic-api -n agentic-api 19000:9000 → ws://localhost:19000/v1/responses
• Model: /models/models/GLM-5.2-FP8

Reproduction

Each response.create envelope has the shape:

{
  "type": "response.create",
  "model": "/models/models/GLM-5.2-FP8",
  "input": [{"role": "user", "content": "<text>"}],
  "max_output_tokens": 512,
  "stream_id": "probe"
}

(plus "store": false and/or "previous_response_id": "<id>" where noted). Read events until response.completed.

Case — cross-connection chaining with store: false (the bug):

1. Open connection A. Send response.create with "store": false and input "Reply with the single word: cherry.". Capture the returned response.id.
2. Close connection A.
3. Open a new connection B. Send response.create with "previous_response_id": "<id from step 1>" and input "What single word did I ask you to say in my previous message? Reply with only that word.".
4. Observed: the model replies cherry — the store: false state was resolved across a new connection.

Expected: the lookup should not resolve (error or no recall), since store: false means nothing is persisted server-side.

Negative control (confirms the id is what carries state, not model guessing):

• Repeat step 3 on a fresh connection with no previous_response_id. The model correctly does not recall the word (it responds that there is no previous message in the session). This rules out the model inferring the answer, confirming that the cross-connection recall in the bug case is genuine server-side persistence keyed on previous_response_id.

Actual results captured

Scenario	store	New connection?	previous_response_id sent?	Result
Baseline (same conn)	true	no	yes	RESOLVED (recalled apple)
Cross-connection	true	yes	yes	RESOLVED (recalled banana)
Cross-connection	false	yes	yes	RESOLVED (recalled cherry) ← bug
Negative control	—	yes	no	NOT resolved (no recall) ✔

Impact

Clients relying on store: false for Zero Data Retention (state held only in memory / connection-local) get server-side persistence they explicitly opted out of, and that state remains retrievable from other connections holding the response id. This both violates the documented store: false contract and is a data-retention concern for ZDR deployments.

Suggested fix

Under store: false, do not persist response/reasoning items server-side, and reject (or refuse to resolve) previous_response_id for such responses — matching the OpenAI Responses API contract, where stateless/ZDR clients replay full history via input.

[LOGO127]

I'd like to take a focused follow-up on connection-local response state and its retention/continuation regression tests, coordinated with #240 rather than duplicating its multiplexing work.

Initial reproduction on main (before the local implementation)

I reproduced this locally against main c04ac196 using the existing Axum/WebSocket test harness, a mock upstream and temporary SQLite (no live model or external API). In a local test-only patch, six of seven contract cases fail; the store:true cross-connection positive control passes:

• A generated store:false response leaves 1 response row and 2 item rows.
• generate:false, store:false leaves 1 response row and 1 item row.
• A fresh connection resolves that unstored response, whether its next request sets store:false or store:true.
• With durable storage disabled, the initial store:false request fails with a persistence error.
• A same-socket store:false child of a stored parent adds another response and two more items.

There are multiple relevant seams: the WebSocket handler forces store=true, generate:false calls persist_turn unconditionally, and the executor's should_persist also treats previous_response_id as a reason to write. Simply removing the handler assignment would not address the full contract.

The current WebSocket documentation also requires in-memory continuation on the same connection, so I would preserve that behavior while preventing durable writes and cross-connection lookup, rather than disabling continuation outright.

Would a follow-up after #240, with a typed core-owned checkpoint scoped to a WebSocket session/lane, be the preferred integration point? I would include prewarm, normal responses, tool-call continuation, cache disposal on disconnect and stored-response controls in the regression scope. The production implementation is not changed yet; this is an AI-assisted investigation and local reproduction, not a claim that the issue is fixed.

---

Local integration update — 2026-09-05

The reproduction above records the initial test-only stage. There is now an unpublished, AI-assisted local candidate, not an upstream fix or an accepted ownership assignment.

• I integrated the core session/group adapter with feat: multiplex response streams over WebSocket #240 at 6f1b1da5 plus main c04ac196 in an isolated worktree. The scheduler, bounded outbound path and resource-limit work from feat: multiplex response streams over WebSocket #240 are preserved. Each named/default lane retains a core member while idle; parent lookup happens at execution start.
• The adapter preserves response-scoped store:false for generated and prewarm requests, with canonical tool/history order, compaction, stored-response controls, fork isolation and disconnect cleanup. A 32 MiB aggregate serialized-checkpoint ceiling counts unique cached/pinned/prepared snapshots and reserves before durable writes. It is a proposed retention bound, not measured process memory or a tuned default.
• A focused overload regression initially failed for both default and named same-lane continuation; the cross-lane fork control passed. The local fix evicts only the rejected lane's matching cached parent. Seven tests now cover request-count and byte-budget overload, malformed routing, fork isolation, active-parent pinning and accepted-work FIFO. An explicit null/invalid stream ID cannot evict the default lane's cache.
• Current transport checks: 72 WebSocket, 70 server-library and 5 HTTP compaction tests pass; workspace all-target Clippy with warnings denied and applicable hooks pass. Core was previously verified in the integrated candidate with 479 tests (3 PostgreSQL-only skips) and 24 public continuation/commit tests. I also prepared an independent core-only review unit on clean main: 454 core / 24 public tests pass, with existing WS/HTTP/split controls. That unit does not activate the WS adapter or close this issue by itself.

One admission-policy detail needs explicit agreement: #240's immediate 429 response is retained, rather than queuing an unbounded rejection work item. Evicting its referenced cached parent can make an earlier accepted but unstarted same-lane continuation return previous_response_not_found after an independent active turn fails. Accepted work still executes FIFO, and an already pinned parent remains usable. This interleaving is now covered by a local regression rather than left implicit.

Would a follow-up on #240's agreed base still be the preferred delivery? Is this immediate-rejection/eviction interaction acceptable, or should that boundary use a different explicitly bounded policy? I can organize the core and transport portions for separate review while keeping the complete state-contract tests together.

Remaining: agree the integration/admission boundary and final review shape, then personal contributor review/DCO before publication. Exact reference active/named-stream limits, stream-ID character/error fidelity and connection expiry are not claimed complete. No live-model/GPU, PostgreSQL execution, full-workspace runtime, packaging or performance acceptance is claimed.

---

Status and validation update — 2026-09-06

The independent core unit is now published as draft #257 at 51c3613. It is core-only: it does not activate the WebSocket adapter or close this issue. The signed commit is unchanged. A separate two-file lease-release-order follow-up is still awaiting personal review/DCO and is not in that PR head.

New local evidence covers the transport/database boundary: the same 74 WebSocket cases passed with PostgreSQL-backed fixtures and with the SQLite control (0 failed/ignored in either run). Disabled-storage controls remain disabled; PostgreSQL cases used 66 fresh UUID-named databases on PostgreSQL 16.15. These use real local WebSocket connections and mocked inference/search upstreams, not a live model.

Source boundary matters: this was an unpublished diagnostic combination of #257, #240 at 6f1b1da5, the adapter, the pending lease follow-up, and only the known conversations.created_at qualification already proposed by the author of #166. That SQL correction is not an original fix of mine and is not included in the standalone core follow-up; without it that candidate still has the two known PostgreSQL upsert failures. This is not validation of the published PR head, all of #166, or upstream CI. Selected-target Clippy, full rustfmt and applicable hooks passed for the diagnostic; no new full-workspace, GPU, real-model, or performance acceptance is claimed.

The integration/admission questions above remain open. The mixed index is unsigned and unpublished, and no ownership or scope approval is assumed. This update records AI-assisted verification only.

[LOGO127]

Now that #240 has merged, thanks for settling the multiplexing base. I'd like to close the remaining integration question before porting the unpublished session adapter onto the merged scheduler (main 7ab6d3b). The core prerequisite remains draft #257; it does not yet change WebSocket behavior.

The earlier diagnostic adapter retained #240's immediate bounded 429 response, and evicted only the rejected lane's matching cached parent. As recorded above, that can make an already accepted but not-yet-started same-lane continuation miss the parent after an independent active turn fails; pinned parents remain usable and accepted work stays FIFO.

Is that rejection/eviction behavior acceptable for the follow-up, or should rejection leave the accepted queue's checkpoint available? I will preserve the merged request/event byte limits and avoid a second scheduler or an unbounded rejection queue. No new transport validation or ownership approval is claimed here; I have not yet ported the adapter onto the merged scheduler. This follow-up is AI-assisted. Thank you for helping pin down the contract.