Skip to content

fix(stream): keep chat SSE alive during long LM Studio generations #1051

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
stablegenius49 wants to merge 1 commit into
base: master
Choose a base branch
from
Open

fix(stream): keep chat SSE alive during long LM Studio generations #1051

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
99 changes: 57 additions & 42 deletions src/app/api/chat/route.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -155,57 +155,72 @@ export const POST = async (req: Request) => {
const responseStream = new TransformStream();
const writer = responseStream.writable.getWriter();
const encoder = new TextEncoder();
const keepAliveMs = 15_000;
let streamClosed = false;
let keepAliveInterval: ReturnType<typeof setInterval> | undefined;

const safeWrite = (payload: Record<string, unknown>) => {
if (streamClosed) return;

try {
writer.write(encoder.encode(JSON.stringify(payload) + '\n'));
Copy link
Contributor

@cubic-dev-ai cubic-dev-ai bot Mar 11, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: writer.write() rejections are not handled; synchronous try/catch in safeWrite does not catch async stream write failures.

Prompt for AI agents 
Check if this issue is valid — if so, understand the root cause and fix it. At src/app/api/chat/route.ts, line 166:

<comment>`writer.write()` rejections are not handled; synchronous `try/catch` in `safeWrite` does not catch async stream write failures.</comment>

<file context>
@@ -155,57 +155,72 @@ export const POST = async (req: Request) => {
+      if (streamClosed) return;
+
+      try {
+        writer.write(encoder.encode(JSON.stringify(payload) + '\n'));
+      } catch (error) {
+        console.warn('Failed to write chat stream payload:', error);
</file context>
Suggested change
writer.write(encoder.encode(JSON.stringify(payload) + '\n'));
void writer
.write(encoder.encode(JSON.stringify(payload) + '\n'))
.catch((error) => {
console.warn('Failed to write chat stream payload:', error);
safeClose();
});
Fix with Cubic

} catch (error) {
console.warn('Failed to write chat stream payload:', error);
}
};

const safeClose = () => {
if (streamClosed) return;

streamClosed = true;

if (keepAliveInterval) {
clearInterval(keepAliveInterval);
}

try {
writer.close();
} catch (error) {
console.warn('Failed to close chat stream:', error);
}
};

keepAliveInterval = setInterval(() => {
Copy link
Contributor

@cubic-dev-ai cubic-dev-ai bot Mar 11, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: Keepalive interval/stream can leak indefinitely when searchAsync rejects without emitting terminal session events.

Prompt for AI agents 
Check if this issue is valid — if so, understand the root cause and fix it. At src/app/api/chat/route.ts, line 188:

<comment>Keepalive interval/stream can leak indefinitely when `searchAsync` rejects without emitting terminal session events.</comment>

<file context>
@@ -155,57 +155,72 @@ export const POST = async (req: Request) => {
+      }
+    };
+
+    keepAliveInterval = setInterval(() => {
+      safeWrite({ type: 'keepAlive' });
+    }, keepAliveMs);
</file context>
Fix with Cubic

safeWrite({ type: 'keepAlive' });
}, keepAliveMs);

safeWrite({ type: 'keepAlive' });

const disconnect = session.subscribe((event: string, data: any) => {
if (event === 'data') {
if (data.type === 'block') {
writer.write(
encoder.encode(
JSON.stringify({
type: 'block',
block: data.block,
}) + '\n',
),
);
safeWrite({
type: 'block',
block: data.block,
});
} else if (data.type === 'updateBlock') {
writer.write(
encoder.encode(
JSON.stringify({
type: 'updateBlock',
blockId: data.blockId,
patch: data.patch,
}) + '\n',
),
);
safeWrite({
type: 'updateBlock',
blockId: data.blockId,
patch: data.patch,
});
} else if (data.type === 'researchComplete') {
writer.write(
encoder.encode(
JSON.stringify({
type: 'researchComplete',
}) + '\n',
),
);
safeWrite({
type: 'researchComplete',
});
}
} else if (event === 'end') {
writer.write(
encoder.encode(
JSON.stringify({
type: 'messageEnd',
}) + '\n',
),
);
writer.close();
safeWrite({
type: 'messageEnd',
});
safeClose();
session.removeAllListeners();
} else if (event === 'error') {
writer.write(
encoder.encode(
JSON.stringify({
type: 'error',
data: data.data,
}) + '\n',
),
);
writer.close();
safeWrite({
type: 'error',
data: data.data,
});
safeClose();
session.removeAllListeners();
}
});
Expand Down Expand Up @@ -234,7 +249,7 @@ export const POST = async (req: Request) => {

req.signal.addEventListener('abort', () => {
disconnect();
writer.close();
safeClose();
});

return new Response(responseStream.readable, {
Expand Down
99 changes: 57 additions & 42 deletions src/app/api/reconnect/[id]/route.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,64 +16,79 @@ export const POST = async (
const responseStream = new TransformStream();
const writer = responseStream.writable.getWriter();
const encoder = new TextEncoder();
const keepAliveMs = 15_000;
let streamClosed = false;
let keepAliveInterval: ReturnType<typeof setInterval> | undefined;

const safeWrite = (payload: Record<string, unknown>) => {
if (streamClosed) return;

try {
writer.write(encoder.encode(JSON.stringify(payload) + '\n'));
Copy link
Contributor

@cubic-dev-ai cubic-dev-ai bot Mar 11, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: safeWrite does not handle async rejection from writer.write, so stream write failures can escape as unhandled promise rejections.

Prompt for AI agents 
Check if this issue is valid — if so, understand the root cause and fix it. At src/app/api/reconnect/[id]/route.ts, line 27:

<comment>`safeWrite` does not handle async rejection from `writer.write`, so stream write failures can escape as unhandled promise rejections.</comment>

<file context>
@@ -16,64 +16,79 @@ export const POST = async (
+      if (streamClosed) return;
+
+      try {
+        writer.write(encoder.encode(JSON.stringify(payload) + '\n'));
+      } catch (error) {
+        console.warn('Failed to write reconnect stream payload:', error);
</file context>
Suggested change
writer.write(encoder.encode(JSON.stringify(payload) + '\n'));
void writer
.write(encoder.encode(JSON.stringify(payload) + '\n'))
.catch((error) => {
console.warn('Failed to write reconnect stream payload:', error);
});
Fix with Cubic

} catch (error) {
console.warn('Failed to write reconnect stream payload:', error);
}
};

const safeClose = () => {
if (streamClosed) return;

streamClosed = true;

if (keepAliveInterval) {
clearInterval(keepAliveInterval);
}

try {
writer.close();
} catch (error) {
console.warn('Failed to close reconnect stream:', error);
}
};

keepAliveInterval = setInterval(() => {
Copy link
Contributor

@cubic-dev-ai cubic-dev-ai bot Mar 11, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: Keepalive interval is started before session.subscribe; if subscribe throws synchronously, the outer catch returns 500 without clearing the interval, leaking a running timer.

Prompt for AI agents 
Check if this issue is valid — if so, understand the root cause and fix it. At src/app/api/reconnect/[id]/route.ts, line 49:

<comment>Keepalive interval is started before `session.subscribe`; if subscribe throws synchronously, the outer catch returns 500 without clearing the interval, leaking a running timer.</comment>

<file context>
@@ -16,64 +16,79 @@ export const POST = async (
+      }
+    };
+
+    keepAliveInterval = setInterval(() => {
+      safeWrite({ type: 'keepAlive' });
+    }, keepAliveMs);
</file context>
Fix with Cubic

safeWrite({ type: 'keepAlive' });
}, keepAliveMs);

safeWrite({ type: 'keepAlive' });

const disconnect = session.subscribe((event, data) => {
if (event === 'data') {
if (data.type === 'block') {
writer.write(
encoder.encode(
JSON.stringify({
type: 'block',
block: data.block,
}) + '\n',
),
);
safeWrite({
type: 'block',
block: data.block,
});
} else if (data.type === 'updateBlock') {
writer.write(
encoder.encode(
JSON.stringify({
type: 'updateBlock',
blockId: data.blockId,
patch: data.patch,
}) + '\n',
),
);
safeWrite({
type: 'updateBlock',
blockId: data.blockId,
patch: data.patch,
});
} else if (data.type === 'researchComplete') {
writer.write(
encoder.encode(
JSON.stringify({
type: 'researchComplete',
}) + '\n',
),
);
safeWrite({
type: 'researchComplete',
});
}
} else if (event === 'end') {
writer.write(
encoder.encode(
JSON.stringify({
type: 'messageEnd',
}) + '\n',
),
);
writer.close();
safeWrite({
type: 'messageEnd',
});
safeClose();
disconnect();
} else if (event === 'error') {
writer.write(
encoder.encode(
JSON.stringify({
type: 'error',
data: data.data,
}) + '\n',
),
);
writer.close();
safeWrite({
type: 'error',
data: data.data,
});
safeClose();
disconnect();
}
});

req.signal.addEventListener('abort', () => {
disconnect();
writer.close();
safeClose();
});

return new Response(responseStream.readable, {
Expand Down
4 changes: 4 additions & 0 deletions src/lib/hooks/useChat.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -551,6 +551,10 @@ export const ChatProvider = ({ children }: { children: React.ReactNode }) => {
const messageId = message.messageId;

return async (data: any) => {
if (data.type === 'keepAlive') {
return;
}

if (data.type === 'error') {
toast.error(data.data);
setLoading(false);
Expand Down