Skip to content

Gap analysis + ported remend conformity corpus - #13

Closed
jonathanKingston wants to merge 2 commits into
mainfrom
claude/streaming-markdown-gap-analysis-ivr0nl
Closed

jonathanKingston wants to merge 2 commits into
mainfrom
claude/streaming-markdown-gap-analysis-ivr0nl

Conversation

@jonathanKingston

@jonathanKingston jonathanKingston commented Jul 5, 2026

Copy link
Copy Markdown
Collaborator

Addresses #12. Two commits: a decision record, and the follow-up code it prescribes.

1. Decision record — docs/streamdown-gap-analysis.md

Written comparison of @copse/streaming-markdown vs Vercel's streamdown and its streaming primitive remend, plus the policy for remend's test suite.

  • They are not the same kind of tool. streamdown is a React component; remend is a string "self-healing" pre-processor; copse is a framework-agnostic renderer. remend competes with one internal step of copse, not the whole thing.
  • Two philosophies. remend heals-then-reparses (append a guessed closing marker, re-parse each token); copse is natively incremental (holds/reveals partial tokens, patches the DOM). copse's hold-and-reveal never commits a wrong intermediate; remend's guess can.
  • Pick copse for a framework-agnostic renderer with high partial-stream fidelity. Caveats recorded: hand-rolled tokenizer to maintain, no math today.

2. Ported conformity corpus — src/remend-corpus.test.ts

The conformity call from #12: adopt remend's input scenarios, not its assertions. remend asserts remend(str) === healedString; copse emits HTML/DOM with pending states, so those fixtures can't run verbatim, and matching remend's expected strings would regress us by design (e.g. remend heals [doc[doc](streamdown:incomplete-link), a fake href; we reveal the label with no href until the real URL arrives — pinned by forbidHtml: [/streamdown:incomplete-link/]).

So each remend input runs through our own invariants:

Suite Asserts
convergence every prefix chunking converges to the fresh full render (chunk-invariance)
no-flash held constructs never leak their raw marker mid-stream
literal-safe lone ~, <, intraword _ stay literal — no tag/entity slip
completed forms once markers close, streamed output equals the at-rest render
known-gap unsupported constructs pinned to current behaviour

31 new tests; full suite 406 green (typecheck + build clean).

Findings (folded into the doc's checklist)

  • Images already handled — forming [alt](partial reveals alt, hides [ + URL, paralleling revealFormingLink. Not a gap.
  • single-tilde / comparison operators / snake_case underscores — all confirmed literal-safe while streaming.
  • ⚠️ KaTeX / $$…$$ math — the one genuine gap. Passes through as literal text; pinned as a known-gap case so implementing math trips the test on purpose. Decision (implement vs. out-of-scope) left open.

Notes

  • New file is test-only + a doc; dist//node_modules/ remain gitignored.
  • Ran npm run typecheck, npm run build, npm test locally — all pass.

🤖 Generated with Claude Code

https://claude.ai/code/session_01R76iwh8Xoc2afkXfx8N4RP

claude added 2 commits July 5, 2026 02:09
Compares @copse/streaming-markdown against Vercel streamdown and its
remend streaming primitive, records why copse is the pick for a
framework-agnostic renderer with high partial-stream fidelity, and sets
the policy for remend's test corpus: adopt the input scenarios, not the
healed-string assertions (asserting copse's own convergence / no-flash
invariants instead). Includes a gap-discovery follow-up checklist.

Refs #12

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R76iwh8Xoc2afkXfx8N4RP
Mirrors the input scenarios from Vercel remend's __tests__ (bold/italic/
code/strike, single-tilde, comparison operators, links, images, snake_case
underscores, blockquote, math) but asserts this renderer's OWN invariants
instead of remend's healed-string outputs:

- convergence: every prefix chunking converges to the fresh full render
- no-flash: held constructs never leak their raw marker mid-stream
- literal-safe: lone ~, <, and intraword _ stay literal (no tag/entity slip)
- completed forms: closed markers render identically to the at-rest render
- known-gap: $$ math is pinned to literal pass-through so adding KaTeX trips it

Findings folded into docs/streamdown-gap-analysis.md: images are already
handled (forming reveal), single-tilde/comparison/underscore all confirmed
safe, and KaTeX is the one genuine gap. 31 new tests, full suite 406 green.

Refs #12

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R76iwh8Xoc2afkXfx8N4RP
@jonathanKingston jonathanKingston changed the title docs: streamdown/remend gap analysis + remend-corpus conformity policy Gap analysis + ported remend conformity corpus Jul 5, 2026

Copy link
Copy Markdown
Collaborator Author

Closing as a duplicate of #14, which addresses the same issue (#12) with a decision record + the ported remend conformity corpus. Consolidating on #14 to avoid two PRs adding the same src/remend-corpus.test.ts file. Reopen if you'd prefer this version instead.


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants