Problem statement / motivation
Let llm-d use agentic-api for Responses state without routing model traffic through the gateway. A stateful turn is exposed as two separately callable steps: hydrate before inference, persist after, so the caller makes the model
call itself.
This approach represents a separation of concerns between agentic-api which is responsible for all state management, hydration, etc., and llm-d coordinator, which is responsible for the AI request and response pipelines.
Proposed solution
hydrate: resolve previous_response_id and return the upstream request with history inlined and continuation and storage fields removed, plus an opaque context for the turn.
persist: take that context and the model's response, store the turn, and return the response envelope.
Both compose the functions the in-process executor already uses, so rehydration and persistence have one
implementation rather than two.
The endpoints are served by a separate crate and binary agentic-llm-d that depends on agentic-server-core and not on the gateway. It serves /internal/hydrate, /internal/persist, /health and /ready.
Alternatives considered
No response
Additional context
No response
Problem statement / motivation
Let
llm-duseagentic-apifor Responses state without routing model traffic through the gateway. A stateful turn is exposed as two separately callable steps:hydratebefore inference,persistafter, so the caller makes the modelcall itself.
This approach represents a separation of concerns between
agentic-apiwhich is responsible for all state management, hydration, etc., andllm-dcoordinator, which is responsible for the AI request and response pipelines.Proposed solution
hydrate: resolveprevious_response_idand return the upstream request with history inlined and continuation and storage fields removed, plus an opaque context for the turn.persist: take that context and the model's response, store the turn, and return the response envelope.Both compose the functions the in-process executor already uses, so rehydration and persistence have one
implementation rather than two.
The endpoints are served by a separate crate and binary
agentic-llm-dthat depends onagentic-server-coreand not on the gateway. It serves/internal/hydrate,/internal/persist,/healthand/ready.Alternatives considered
No response
Additional context
No response