fix(stream): keep chat SSE alive during long LM Studio generations

[stablegenius49]

Summary

• send lightweight keepAlive frames every 15s on chat and reconnect SSE streams
• emit an immediate keepalive frame so long-running LM Studio requests don't start with a completely idle response
• ignore keepalive frames in the chat client while preserving existing block/update/error handling

Why

Long-running LM Studio generations can leave the /api/chat response completely idle for minutes before the first real block arrives. That makes it easier for the browser/proxy side to tear down the SSE connection, which in turn aborts the upstream model request and shows up in LM Studio as the client disconnecting mid-generation.

Keeping the SSE stream warm avoids that idle gap without changing message semantics.

Closes #1030

Testing

• ./node_modules/.bin/tsc --noEmit

---

Summary by cubic

Keep SSE chat streams alive during long LM Studio generations by sending lightweight keepAlive frames and making stream writes/closes safe. Prevents browser/proxy timeouts and upstream model aborts. Addresses #1030.

• Bug Fixes
  • Send keepAlive every 15s and immediately on start for /api/chat and /api/reconnect/[id].
  • Add safe write/close helpers with interval cleanup to avoid stream errors and leaks.
  • Ignore keepAlive frames in useChat so existing block/update/error handling stays unchanged.

^{Written for commit 051ea77. Summary will update on new commits.}

[cubic-dev-ai]

4 issues found across 3 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="src/app/api/reconnect/[id]/route.ts">

<violation number="1" location="src/app/api/reconnect/[id]/route.ts:27">
P2: `safeWrite` does not handle async rejection from `writer.write`, so stream write failures can escape as unhandled promise rejections.</violation>

<violation number="2" location="src/app/api/reconnect/[id]/route.ts:49">
P2: Keepalive interval is started before `session.subscribe`; if subscribe throws synchronously, the outer catch returns 500 without clearing the interval, leaking a running timer.</violation>
</file>

<file name="src/app/api/chat/route.ts">

<violation number="1" location="src/app/api/chat/route.ts:166">
P2: `writer.write()` rejections are not handled; synchronous `try/catch` in `safeWrite` does not catch async stream write failures.</violation>

<violation number="2" location="src/app/api/chat/route.ts:188">
P2: Keepalive interval/stream can leak indefinitely when `searchAsync` rejects without emitting terminal session events.</violation>
</file>

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.}

[cubic-dev-ai]

P2: Keepalive interval is started before session.subscribe; if subscribe throws synchronously, the outer catch returns 500 without clearing the interval, leaking a running timer.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At src/app/api/reconnect/[id]/route.ts, line 49:

<comment>Keepalive interval is started before `session.subscribe`; if subscribe throws synchronously, the outer catch returns 500 without clearing the interval, leaking a running timer.</comment>

<file context>
@@ -16,64 +16,79 @@ export const POST = async (
+      }
+    };
+
+    keepAliveInterval = setInterval(() => {
+      safeWrite({ type: 'keepAlive' });
+    }, keepAliveMs);
</file context>

[cubic-dev-ai]

P2: safeWrite does not handle async rejection from writer.write, so stream write failures can escape as unhandled promise rejections.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At src/app/api/reconnect/[id]/route.ts, line 27:

<comment>`safeWrite` does not handle async rejection from `writer.write`, so stream write failures can escape as unhandled promise rejections.</comment>

<file context>
@@ -16,64 +16,79 @@ export const POST = async (
+      if (streamClosed) return;
+
+      try {
+        writer.write(encoder.encode(JSON.stringify(payload) + '\n'));
+      } catch (error) {
+        console.warn('Failed to write reconnect stream payload:', error);
</file context>

Suggested change

	writer.write(encoder.encode(JSON.stringify(payload) + '\n'));
	void writer
	.write(encoder.encode(JSON.stringify(payload) + '\n'))
	.catch((error) => {
	console.warn('Failed to write reconnect stream payload:', error);
	});

[cubic-dev-ai]

P2: Keepalive interval/stream can leak indefinitely when searchAsync rejects without emitting terminal session events.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At src/app/api/chat/route.ts, line 188:

<comment>Keepalive interval/stream can leak indefinitely when `searchAsync` rejects without emitting terminal session events.</comment>

<file context>
@@ -155,57 +155,72 @@ export const POST = async (req: Request) => {
+      }
+    };
+
+    keepAliveInterval = setInterval(() => {
+      safeWrite({ type: 'keepAlive' });
+    }, keepAliveMs);
</file context>

[cubic-dev-ai]

P2: writer.write() rejections are not handled; synchronous try/catch in safeWrite does not catch async stream write failures.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At src/app/api/chat/route.ts, line 166:

<comment>`writer.write()` rejections are not handled; synchronous `try/catch` in `safeWrite` does not catch async stream write failures.</comment>

<file context>
@@ -155,57 +155,72 @@ export const POST = async (req: Request) => {
+      if (streamClosed) return;
+
+      try {
+        writer.write(encoder.encode(JSON.stringify(payload) + '\n'));
+      } catch (error) {
+        console.warn('Failed to write chat stream payload:', error);
</file context>

Suggested change

	writer.write(encoder.encode(JSON.stringify(payload) + '\n'));
	void writer
	.write(encoder.encode(JSON.stringify(payload) + '\n'))
	.catch((error) => {
	console.warn('Failed to write chat stream payload:', error);
	safeClose();
	});