[Bug]: Streaming Endpoint Double-Yields Answer: Raw Tokens Followed by Full Framed Answer

[hariom888]

Summary

Description

In _generate_and_stream() inside ask_question_stream (rag-service/main.py), the Groq API streaming loop yields each token as it arrives:

yield _sse_frame(token)           # line ~4066 — individual tokens streamed

After the loop finishes, the code re-applies apply_mode_framing to full_answer and then yields the entire framed answer a second time:

yield _sse_frame(framed)          # line ~4090 — full answer yielded again

followed by yield _sse_done().

Affected Files

• rag-service/main.py — _generate_and_stream() inner function inside ask_question_stream, specifically the yield _sse_frame(framed) call after the Groq streaming loop

Steps to reproduce

The post-loop yield _sse_frame(framed) must be removed. Instead, the framing delta (i.e., only the parts of framed that differ from full_answer) or a structured metadata SSE event (e.g., event: sources) should be yielded separately so the frontend can attach citation data without re-rendering the answer body. The [DONE] frame should remain as the terminal marker.

Expected behavior

Any SSE consumer that renders all received frames will display the answer once in token-by-token form and then a second time as a complete block, resulting in visually duplicated output. Consumers that look for [DONE] to stop rendering will display both emissions before stopping.

Actual behavior

• All users of the streaming endpoint see the answer text duplicated in the UI.
• The framed answer (which may differ from the raw concatenation due to mode transformations like citation injection) appears after the token stream, creating inconsistent content.
• The citation_sources list is only emitted with the second frame, not with individual tokens, so the frontend citation-rendering logic may receive mismatched source lists.

Additional context

No response

[hariom888]

Hi — requesting GSSoC'26 assignment.

Technical understanding: The Groq streaming loop (lines ~4045–4073) yields every token delta. The post-loop block at lines ~4076–4093 was intended to emit only framing metadata (citations, mode-framed wrapper) after streaming completes, but instead re-yields the complete answer text via yield _sse_frame(framed). The [DONE] marker then follows immediately after, so clients correctly stop — but not before displaying both the incremental token stream and the repeated complete answer.

Affected modules/files:

• rag-service/main.py — _generate_and_stream() closure (~line 4076)

Proposed fix approach: The post-loop yield _sse_frame(framed) must be removed. Instead, the framing delta (i.e., only the parts of framed that differ from full_answer) or a structured metadata SSE event (e.g., event: sources) should be yielded separately so the frontend can attach citation data without re-rendering the answer body. The [DONE] frame should remain as the terminal marker.

Regression/unit test considerations:

• Add a test that collects all SSE frames from /ask/stream, concatenates data: payloads (excluding [DONE]), and asserts the answer text appears exactly once.
• Add a test verifying the answer text in the concatenated stream equals full_answer (before mode framing), not framed.