design(streaming): clarify session-scoped runtime ownership for running WebUI sessions

[dso2ng]

Context

Several recent fixes have improved WebUI behavior around running sessions, stream reattachment, reload recovery, and multi-tab streaming:

• Improve reliability of in-flight session switching and stream reattachment #1466 framed the core invariant for in-flight session switching:

  The active pane is only a projection; running state belongs to the session that owns the stream.

• fix: reuse inflight session stream #1467 reused the existing EventSource for the same (session_id, stream_id) instead of tearing it down during sidebar switch-back.
• Fix batch session actions and in-flight reload recovery #1473 improved normal reload recovery for in-flight / zero-message sessions.
• fix: restore inflight session on bfcache pageshow #1480 aligned BFCache pageshow restore with the normal reload recovery path.
• fix: add sidebar cancel for running sessions #1493 made sidebar cancellation use the row session's active_stream_id, not the active pane global.
• (bug) Multiple browser sessions #1584 identified multi-tab stream splitting as a destructive shared-queue problem.
• fix: broadcast SSE stream events to multiple tabs #1598 fixed that transport layer with fan-out / per-subscriber queues.
• Crash recovery: WAL design + integration for in-flight assistant tokens #1585 tracks the separate crash-recovery / WAL direction for replaying already-emitted in-flight output after process loss.

These changes fixed important concrete bugs, but they also expose a broader architectural question: which runtime state is pane-owned, which runtime state is session-owned, and what invariants should future PRs preserve?

If this direction matches the intended architecture, this issue could also serve as a tracking issue for small follow-up PRs.

Proposed mental model

The WebUI appears to have two different concepts that should stay separate:

1. Active pane state

   • S.session
   • S.messages
   • S.busy
   • S.activeStreamId
   • currently rendered approval / clarify / composer UI

2. Running session state

   • INFLIGHT[session_id]
   • LIVE_STREAMS[session_id]
   • server-side active_stream_id
   • pending user message / checkpoint state
   • approval / clarify ownership
   • sidebar row runtime metadata
   • background completion / canonical-session rotation

The active pane is a projection of one session. A running turn belongs to the session that owns the stream, even when that session is not the currently viewed pane.

Why this matters

Long-running Hermes turns may continue while the user:

• switches to another sidebar session;
• opens the same session in another tab;
• reloads /session/<id>;
• restores from BFCache;
• opens the root / page in a new tab;
• receives approval / clarify prompts in the background;
• cancels a running session from the sidebar;
• hits network drops or EventSource reconnect paths;
• eventually needs crash recovery / replay behavior.

If code paths rely too heavily on active-pane globals, several classes of bugs can reappear:

• switching panes tears down or duplicates a still-running stream;
• a background session completion mutates the active pane;
• cancel / approval / clarify actions target the wrong session;
• a root tab projects into a saved running session and appears globally blocked;
• S.busy prevents unrelated pane actions even though the running turn belongs to another session;
• transport fan-out and runtime ownership get conflated with replay / WAL durability.

Suggested invariants

I would like to confirm whether the following invariants match the intended direction:

1. S.session, S.messages, S.busy, and S.activeStreamId should describe the currently viewed pane only.
2. Running state should be keyed by session_id / stream_id, not by whichever pane is currently active.
3. Returning to the same running session should reuse the existing live transport when possible.
4. Switching away from a running session should not inherently close, hide, or delete that session's runtime state.
5. Background completion should refresh sidebar / alias / canonical-session metadata without mutating an unrelated active pane.
6. Approval and clarify state should be owned by the session that requested it, and rendered only when that session is active.
7. Sidebar row actions should use row-owned runtime metadata, such as session.active_stream_id, not active-pane globals.
8. Multi-tab live streaming should be handled by stream fan-out; replay / crash recovery should remain a separate WAL or snapshot design.
9. Root / boot behavior should be considered separately from explicit /session/<id> reload / reattach behavior, especially when the saved session is currently running.
10. Future refactors should preserve the current no-build-step / vanilla JS architecture and land as small, reviewable PRs.

Possible follow-up PR ladder

If this direction is correct, I think the safest path is not a large runtime rewrite, but a small PR ladder. Examples:

1. Add/expand regression coverage for remaining running-session ownership paths.
2. Audit terminal handlers so completion / cancel / error cleanup is scoped to the owning session.
3. Audit approval / clarify rendering so pending state survives switching away and reappears when returning.
4. Clarify root / boot policy for saved running sessions vs explicit /session/<id> reload recovery.
5. Ensure background canonical-session rotation updates sidebar / aliases even when the completed session is not active.
6. Only after the invariants are pinned, consider centralizing per-session runtime state behind a small helper object.

A possible future shape might be:

SESSION_RUNTIME[sid] = {
  streamId,
  status,
  eventSource,
  messages,
  toolCalls,
  approvalState,
  clarifyState,
  canonicalSessionId,
  lastEventAt,
};

But this issue is not proposing a broad rewrite up front. The immediate goal is to confirm the ownership model and use it to guide narrow fixes.

Questions

1. Does the active-pane vs session-owned runtime model above match the intended architecture?
2. Are there any current globals that should intentionally remain app-wide rather than session-scoped?
3. Should root / restore a saved running session automatically, or should explicit /session/<id> be the only path that projects into a running session on boot?
4. Should late subscribers rely only on fan-out from their attach point, or should a compact live-progress snapshot be considered separately from full WAL replay?
5. Is Crash recovery: WAL design + integration for in-flight assistant tokens #1585 the right place to keep crash-recovery / replay design, while this issue tracks active-pane/runtime ownership?
6. Would maintainers prefer follow-up PRs under one tracking issue, or individual bug reports for each remaining scenario?

[nesquena-hermes]

Thanks for the framing @dso2ng — this is a genuinely useful design write-up. The invariant you've stated ("running state belongs to the session that owns the stream, not the active pane") is what's been driving the recent in-flight session work (#1466, #1467, #1473), and writing it down explicitly is the right next step.

Filing under M2 (Power user features) with tracking since this is a design discussion rather than a single bug — work that comes out of it will be filed as separate sub-issues.

Two questions on the proposal:

1. Lifecycle of the per-session runtime handle. Today the (session_id, stream_id) pair is the natural identifier, but stream_id rotates per turn. Are you proposing the runtime handle be keyed by session_id alone (with one runtime that may host multiple sequential streams) or by (session_id, stream_id) (handle is per-stream and replaced each turn)? The first is simpler for tab-switch UX; the second is closer to how the SSE layer currently works.

2. Cross-tab arbitration. Two tabs open to the same session, both subscribe to the same stream. Today this works (fix: reuse inflight session stream #1467) because the EventSource is reused. If we move to a runtime-handle model, do both tabs share a single runtime, or do they each get a passive subscriber view of one runtime owned by the originating tab?

If you have time to sketch out where the boundary lives (api/streaming.py vs. api/models.py vs. a new module), that would unblock the design discussion and let me file the implementation sub-issues against it.

[dso2ng]

Thanks - that framing matches how I was thinking about the boundary after reading the current code.

I would not make this a pure session_id vs (session_id, stream_id) choice. I think there are two layers:

1. Transport / execution layer: stream-keyed.
   STREAMS[stream_id], CANCEL_FLAGS[stream_id], AGENT_INSTANCES[stream_id], partial-text/reasoning/tool-call buffers, and /api/chat/stream?stream_id=... are naturally per-turn / per-stream. A stream_id rotates each turn, and those resources should be created and cleaned up with that stream.

2. Session runtime / UI ownership layer: session-keyed, with a current stream pointer.
   The WebUI concept that survives tab switches is "session X is running / needs attention / has current stream Y". That should be keyed by session_id, with currentStreamId / active_stream_id as a field that rotates per turn.

So the shape I would aim for is roughly:

SESSION_RUNTIME[session_id] = {
  currentStreamId,
  status,
  approvalState,
  clarifyState,
  canonicalSessionId,
  lastEventAt,
}

while the server transport remains:

STREAMS[stream_id]

For cross-tab behavior, I would avoid making the originating tab the owner. Once a stream exists, the server-side stream/channel is the owner of live transport state, and each tab is a passive subscriber/projection. Two tabs on the same session should subscribe to the same stream_id via fan-out; each tab may keep its own active-pane cache, but no tab should be required to stay alive for the runtime to remain valid.

A boundary sketch:

• api/streaming.py: per-stream execution, event emission, cancel/error/done cleanup.
• api/config.py::StreamChannel: fan-out for one stream_id; transport-level only, not durable session runtime.
• api/models.py / session metadata: durable/current session runtime markers such as active_stream_id, pending user/checkpoint state, and canonical session metadata.
• static/messages.js / static/sessions.js: per-tab projection and session-keyed browser runtime cache (INFLIGHT[session_id], LIVE_STREAMS[session_id]), with streamId as the current transport pointer.

I would probably not introduce a new backend module first. I'd first land small sub-issues/PRs that preserve this boundary, then only factor a helper/module once repeated code makes the boundary obvious.

[nesquena-hermes]

Summary

This boundary sketch matches the actual code more closely than I would have guessed. The two-layer split you've drawn — STREAMS[stream_id] for transport, SESSION_RUNTIME[session_id] for ownership/UI state — is what's already in place on the server, and on the client there are session-keyed INFLIGHT and LIVE_STREAMS maps with S.activeStreamId as the rotating transport pointer. So most of this proposal is "preserve the boundary that's already there and don't blur it" rather than a new architecture.

Code reference — what already matches

Server transport, exactly as you described, in api/streaming.py:23:

from api.state_sync import (
    STREAMS, STREAMS_LOCK, CANCEL_FLAGS, AGENT_INSTANCES, STREAM_PARTIAL_TEXT,
    ...
)

All four of those are stream-id-keyed and they're cleared together when a stream finishes (api/streaming.py:3063-3066):

with STREAMS_LOCK:
    STREAMS.pop(stream_id, None)
    CANCEL_FLAGS.pop(stream_id, None)
    AGENT_INSTANCES.pop(stream_id, None)

s.active_stream_id is the rotating session-level pointer (api/streaming.py:2571,2677,3032), set on stream start and cleared on done/error. That matches your "session-keyed with a current stream pointer" layer.

Client session-keyed runtime, in static/messages.js:306-313:

const LIVE_STREAMS={};

function closeLiveStream(sessionId, streamId){
  const live=LIVE_STREAMS[sessionId];
  if(!live) return;
  if(streamId&&live.streamId!==streamId) return;
  try{live.source.close();}catch(_){ }
  delete LIVE_STREAMS[sessionId];
}

INFLIGHT[activeSid] (the per-session messages/toolCalls cache) is the parallel browser-side projection. S.activeStreamId is the current-stream pointer in S (the active-pane focus), and S.session.active_stream_id is the durable session-state mirror.

Code reference — fan-out already done

Your "passive subscriber/projection" model for cross-tab is implemented in api/config.py:3123 (StreamChannel):

class StreamChannel:
    """Broadcast SSE events to every connected browser tab for a stream.

    While no tab is connected, events are buffered so the first/reconnected
    subscriber still receives the stream tail that arrived during the gap.
    Once one or more subscribers are attached, new events are broadcast to all
    of them instead of being consumed destructively by a single queue reader.
    """

subscribe()/unsubscribe() give per-tab queues; put_nowait() broadcasts to all live subscribers and otherwise buffers. So the originating-tab-as-owner failure mode you're warning against can't actually happen with the current StreamChannel — any tab can subscribe, any tab can drop, and the stream stays alive as long as the agent thread is writing to it.

Where the boundary still leaks

A couple of places I think still warrant the design clarification you're proposing:

1. active_stream_id is the only durable runtime field on the session. Things like approval state, clarify state, and "needs attention" badges are derived ad-hoc by static/sessions.js and static/messages.js from inflight inferences. A SESSION_RUNTIME[session_id] registry on the server (status, approvalState, clarifyState, lastEventAt) would make those a single source of truth instead of scattered derivations. There is already a partial server-side analog (SESSION_AGENT_CACHE mentioned in api/config.py) used for agent reuse, but it isn't the broader runtime map you're describing.

2. Reload/reattach uses active_stream_id from the session row. api/streaming.py:3133 reads getattr(s, "active_stream_id", None) to decide whether to reconnect. That works because the session object is the durable carrier, but it conflates "this session has a live stream" with "this session has unfinished UI state needing attention" — they're the same today only because the only state we track is "is the stream still in STREAMS".

3. Browser cache keys aren't normalized. INFLIGHT is keyed by activeSid; LIVE_STREAMS is keyed by sessionId; a few code paths use S.session.session_id directly. A small accessor like getSessionRuntime(sid) that always reads from one map would make the boundary explicit and would catch the subtle "wrong tab key" bugs that show up when active-pane state and session-state get tangled.

Agree with your "land sub-issues first, factor a helper module only when the duplication is obvious" approach. Worth a follow-up issue or two for (1) and (3) specifically — (1) feels like the highest-leverage one if WAL design (#1585) lands and starts wanting to persist runtime state across crashes.