Skip to content

Commit 82dfd60

Browse files
committed
docs: streaming-markdown vs remend/streamdown gap analysis + conformity policy
Adds a decision record under docs/decisions/ capturing the gap analysis vs. Vercel's streamdown/remend, the "adopt inputs not assertions" conformity policy for the remend test corpus, the deliberate divergences (label-only forming links, HTML escaped by design, no math today), and the follow-up checklist with per-item status. Conformance figures corrected against the live baseline (510/652 overall, 510/588 in-scope excluding the 64 by-design HTML examples). Wires the checklist as real additive tests in src/remend-corpus.test.ts: the remend streaming edge-case inputs (single tilde, comparison operators, underscore identifier, forming image, forming link) are fed through the existing convergence/no-flash machinery and asserted against copse's own invariants — (a) no marker flashes as structural markup in any prefix frame, (b) the committed render equals the static renderMarkdown render, (c) every prefix converges to the same fresh full render. KaTeX/math documented as a known gap. Docs + additive tests only; no parser or baseline changes. typecheck and the full suite (397 tests) stay green. Closes #12 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Cv1iyJb9PHMFG3YGp6ffkT
1 parent fb1a5bf commit 82dfd60

2 files changed

Lines changed: 363 additions & 0 deletions

File tree

Lines changed: 167 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,167 @@
1+
# 0001 — `@copse/streaming-markdown` vs. remend/streamdown, and a conformity policy for the remend corpus
2+
3+
Status: accepted · Relates to [#12](https://github.com/copse-dev/streaming-markdown/issues/12)
4+
5+
Decision record capturing how this package compares to Vercel's
6+
[streamdown](https://github.com/vercel/streamdown) and its streaming primitive
7+
[remend](https://github.com/vercel/streamdown/tree/main/packages/remend), what we
8+
adopt from remend's test suite, and which divergences are deliberate. For the
9+
design invariants and the conformance baseline this references, see
10+
[ARCHITECTURE.md](../ARCHITECTURE.md).
11+
12+
## The framing that matters
13+
14+
These are not the same kind of tool, so a flat "A vs B" is the wrong axis:
15+
16+
| | **copse/streaming-markdown** | **remend** | **streamdown** |
17+
| ---------------------- | ---------------------------------------------- | ------------------------------------------------ | --------------------------------------- |
18+
| What it is | Full renderer (markdown → HTML/DOM) | String **pre-processor** ("self-healing") | React **component** |
19+
| Streaming strategy | Native incremental state machine — holds/reveals partial tokens via designed pending states | Append synthetic closing markers, then re-parse the whole string | remend + remark/rehype, re-render each token |
20+
| Framework | Host-independent | Framework-agnostic (healer only) | React-bound |
21+
| Deps | `entities`, `dompurify`, `highlight.js` | **zero** | React + remark + rehype + shiki + katex + mermaid |
22+
| Streaming-layer output | HTML string **or** incremental DOM patches | a *healed markdown string* (still needs a renderer) | rendered React tree |
23+
24+
**Two philosophies:**
25+
26+
- **Heal-then-reparse (remend):** cheap, portable, but the completion is an
27+
*optimistic guess* — it inserts a marker the model never sent and re-parses the
28+
whole document every token. A transient frame can show a completion that later
29+
changes.
30+
- **Native incremental (copse):** more code, but transient frames are engineered
31+
(hold-and-reveal) rather than a side-effect of auto-closing, and the DOM is
32+
patched incrementally instead of re-parsed O(n) per token.
33+
34+
## Gap analysis
35+
36+
**Where copse is ahead**
37+
38+
- Genuinely renderer-complete **and** host-independent. streamdown gives nothing
39+
outside React; remend gives only the string-heal step (you still build the
40+
renderer). copse is the whole pipeline with an injectable `LinkDecorator`.
41+
- **Measurable CommonMark conformance** — the ~650-example spec suite with a
42+
pinned baseline. remend's healing is heuristic; streamdown's final fidelity is
43+
whatever remark-gfm does, ungated by a spec harness. (Live numbers below.)
44+
- **Convergence guarantee under incremental patching** —
45+
`streaming-convergence.test.ts` fuzzes every prefix cut and asserts streaming
46+
any chunking converges to the byte-identical fresh render *while patching the
47+
DOM incrementally*. remend/streamdown get convergence trivially by re-rendering
48+
from scratch each token; copse proves it holds without the reparse.
49+
- Two emitters (string HTML + incremental DOM) with a benchmark harness that
50+
catches super-linear regressions.
51+
52+
**Where streamdown/remend is ahead**
53+
54+
- **KaTeX / math** — copse has none; remend explicitly heals `$$…`. A real
55+
feature gap (see the follow-up table below — documented as a known gap for now).
56+
- Shiki highlighting (finer-grained than highlight.js) and a polished
57+
Tailwind/shadcn drop-in *if you are already a React app*.
58+
- remend-the-primitive is smaller than copse-core (zero deps vs. `dompurify` +
59+
`highlight.js`).
60+
- Larger community / vendor backing.
61+
62+
### Conformance numbers (live baseline)
63+
64+
The headline figures in the issue predate the current baseline. From
65+
`tests/fixtures/commonmark/conformance-baseline.json` (`summaryBySection`) at the
66+
time of writing:
67+
68+
- **510 / 652** official spec examples pass at rest (~78%).
69+
- Two sections fail **by design** because the renderer escapes untrusted HTML
70+
rather than passing it through (sanitize-at-the-sink): **HTML blocks 2/44** and
71+
**Raw HTML 8/20**. Excluding those **64 HTML examples**, the in-scope ceiling is
72+
**588 examples**, of which **510 pass (~87%)**.
73+
74+
So the honest phrasing is "structural CommonMark minus raw-HTML passthrough, by
75+
security choice." Treat these figures as approximate and read `summaryBySection`
76+
in the baseline JSON for the live per-section counts; they move as non-HTML
77+
conformance grows. Do **not** re-baseline in this doc's PR.
78+
79+
## Recommendation: which to pick
80+
81+
For a **framework-agnostic renderer with high partial-stream fidelity → copse**,
82+
clearly. streamdown is a candidate only if you are already all-in on React and
83+
happy to ship shiki + katex + mermaid + remark. remend alone is not a competitor
84+
to copse — it competes with *one internal step* of copse (the "don't flash raw
85+
syntax" logic), and copse's hold-and-reveal approach to that step is
86+
architecturally stronger than remend's guess-and-append because it never commits a
87+
wrong intermediate.
88+
89+
Honest caveats: copse costs a hand-rolled tokenizer to maintain, and it lacks math
90+
today.
91+
92+
## Conformity policy: adopt inputs, not assertions
93+
94+
**We adopt remend's input corpus; we do not adopt its assertions.**
95+
96+
remend's tests assert `remend(inputString) === healedString` — a claim about
97+
*which closing markers to append to a raw string*. copse does not emit a healed
98+
markdown string; it emits HTML/DOM with pending states. **The output types do not
99+
match, so the fixtures cannot run verbatim as pass/fail.**
100+
101+
Worse, conforming to remend's *expected outputs* would regress copse by design.
102+
Example: remend turns `[documentation` into
103+
`[documentation](streamdown:incomplete-link)` — a fake href. copse's
104+
`revealFormingLink` (`render-pending-line.ts`) instead shows just the label text
105+
with **no** href until the real URL arrives (never a bogus/partial/dead link).
106+
Matching remend's string would break a copse invariant.
107+
108+
**The right move:**
109+
110+
1. Mine remend's `__tests__` as a checklist of streaming edge-case **inputs**, fed
111+
into copse's existing `streaming-convergence` / `streaming-pending-matrix`
112+
harness.
113+
2. Assert copse's **own** invariants on each: (a) no raw marker flashes as
114+
structural markup in any prefix frame, (b) once the input commits, the streamed
115+
render equals the static `renderMarkdown` render, (c) every prefix converges to
116+
the same fresh full render.
117+
3. Use it for **gap discovery** — their suite surfaces constructs copse may not
118+
cover.
119+
120+
This is implemented in [`src/remend-corpus.test.ts`](../../src/remend-corpus.test.ts).
121+
122+
## Deliberate divergences (do not "fix" these later by mistake)
123+
124+
- **Label-only forming links.** A forming link/image reveals only its label text
125+
with no `href`/`src` until the real destination arrives. remend emits a
126+
placeholder href (`streamdown:incomplete-link`); copse deliberately never
127+
renders a dead/partial link. Enforced by `revealFormingLink`
128+
(`render-pending-line.ts`) and asserted in `streaming-link-label.test.ts` and
129+
the new corpus test.
130+
- **Raw HTML escaped by design.** Untrusted HTML is escaped, not passed through
131+
(sanitize-at-the-sink). This is why HTML blocks (2/44) and Raw HTML (8/20) cap
132+
out; a benign attribute-less inline allowlist (`b i u s del ins sub sup kbd mark
133+
br`) is the only passthrough. See ARCHITECTURE "Raw-HTML policy".
134+
- **No math today.** `$$…$$` / KaTeX is unsupported and is a documented known gap,
135+
not a regression.
136+
137+
## Follow-up checklist (gap discovery from the remend corpus)
138+
139+
Every item below is either wired as a real test in `src/remend-corpus.test.ts`
140+
(feeding the input through copse's convergence / no-flash machinery) or documented
141+
here as a deliberate known gap.
142+
143+
| Item | Status |
144+
| --- | --- |
145+
| Port the remend streaming input corpus into a convergence/no-flash test (inputs only, copse invariants as assertions). | **Done** — `src/remend-corpus.test.ts` asserts invariants (a) no marker flash, (b) committed == static, (c) prefix convergence. |
146+
| KaTeX / `$$…$$` math — decide known-gap vs. implement. | **Known gap** — documented above; not implemented in this PR. |
147+
| Single tilde (`20~25`) stays literal while streaming. | **Covered** — no `<del>`/`<s>` in any frame; a half-open trailing `~` is held, the full input reveals `~` literally. |
148+
| Comparison operators (`20 < 30`) — no spurious tag/entity mid-stream. | **Covered** — `<` stays escaped; no `a`/`em`/`strong`/`del` element in any frame. |
149+
| Images — forming `![alt](partial` / `[alt](partial` reveal gracefully. | **Covered** — no `<img>`/`<a>`, no partial `src`/destination in any frame; label revealed. |
150+
| Underscore-in-identifier (`foo_bar_baz`) not italicised mid-stream. | **Covered** — no `<em>`/`<strong>` in any frame; text stays literal. |
151+
| Document the divergence policy (label-only forming links vs. remend's placeholder href). | **Done** — see "Deliberate divergences" above. |
152+
153+
## Rationale sharpening (for whoever writes the pitch)
154+
155+
- **"Not bound to React"** — strongest point: copse is the only framework-agnostic
156+
*renderer* of the three.
157+
- **"CommonMark + GFM compat"** — true and measured, but state the deliberate cap:
158+
copse escapes raw HTML by design (2/44 HTML-blocks, 8/20 raw-inline), so it is
159+
"structural CommonMark minus raw-HTML passthrough, by security choice."
160+
- **"100% partial-stream fidelity"** — name it precisely: **chunk-invariant
161+
convergence** (`streaming-convergence.test.ts`) — streaming any chunking yields
162+
the byte-identical static render, proven under incremental DOM patching (not the
163+
free version you get from re-rendering everything).
164+
- **"Small — x kb over y kb"** — weakest bullet; needs real gzipped `dist`
165+
numbers. True vs. streamdown-the-product (React + shiki + katex + mermaid),
166+
**false** vs. remend-the-primitive (zero-dep). Be explicit about which competitor
167+
and which config, or drop it.

‎src/remend-corpus.test.ts‎

Lines changed: 196 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,196 @@
1+
// Gap-discovery corpus mined from remend's `__tests__` (Vercel/streamdown).
2+
//
3+
// We adopt remend's streaming edge-case INPUTS, not its assertions: remend
4+
// asserts `remend(input) === healedString` (which closing markers to append to
5+
// a raw string), a claim about an output type copse does not produce. copse
6+
// emits HTML/DOM with engineered pending states, so instead of matching a
7+
// healed string we assert copse's OWN invariants on each input:
8+
// (a) no raw markdown marker "flashes" as structural markup in any prefix frame,
9+
// (b) once the input commits, the streamed render equals the static render,
10+
// (c) every prefix converges to the same fresh full streamed render.
11+
// See docs/decisions/0001-streaming-markdown-vs-remend-streamdown.md.
12+
import '../tests/setup-dom-jsdom.ts'
13+
import { describe, it } from 'node:test'
14+
import assert from 'node:assert/strict'
15+
import { renderMarkdown } from './renderer.ts'
16+
import { sanitizeRenderedMarkdown } from './sanitize.ts'
17+
import {
18+
renderStreamingMarkdown,
19+
splitForStreaming,
20+
StreamingMarkdownRenderer,
21+
} from './streaming.ts'
22+
23+
/** Visible streaming HTML: committed blocks + any forming table + live tail. */
24+
function extractStreamingDisplay(host: HTMLElement): string {
25+
const parts: string[] = []
26+
const complete = host.querySelector('.stream-complete')
27+
if (complete) parts.push(complete.innerHTML)
28+
const forming = host.querySelector('.stream-forming')
29+
if (forming instanceof HTMLElement && !forming.hidden) parts.push(forming.innerHTML)
30+
const pending = host.querySelector('.stream-pending')
31+
if (pending instanceof HTMLElement && !pending.hidden && pending.innerHTML !== '') {
32+
parts.push(pending.innerHTML)
33+
}
34+
return parts.join('')
35+
}
36+
37+
function streamingDisplayAfterUpdates(markdown: string, cuts: number[]): string {
38+
const host = document.createElement('div')
39+
const renderer = new StreamingMarkdownRenderer(host)
40+
for (const cut of cuts) {
41+
renderer.update(markdown.slice(0, cut))
42+
}
43+
return extractStreamingDisplay(host)
44+
}
45+
46+
/** Every prefix length — the corpus inputs are short, so exhaustive is cheap. */
47+
function everyPrefix(text: string): number[] {
48+
return Array.from({ length: text.length + 1 }, (_, i) => i)
49+
}
50+
51+
/** Parse a streamed HTML frame into a detached element for structural queries. */
52+
function frameElement(html: string): HTMLElement {
53+
const div = document.createElement('div')
54+
div.innerHTML = html
55+
return div
56+
}
57+
58+
interface CorpusCase {
59+
/** remend `__tests__` file this input is mined from. */
60+
readonly source: string
61+
readonly input: string
62+
/**
63+
* Structural markup that must never appear in any prefix frame — the marker
64+
* would be a mid-stream "flash" copse deliberately holds back.
65+
*/
66+
readonly forbiddenSelectors: string
67+
/** Substrings that must never appear in any prefix frame (e.g. a partial URL). */
68+
readonly forbiddenText?: readonly RegExp[]
69+
/**
70+
* When true, the marker stays literal text end to end: every prefix frame's
71+
* visible text is a prefix of the raw input (a half-open trailing marker may
72+
* be *held*, never turned into markup or extra characters), and the full
73+
* input reveals the marker verbatim.
74+
*/
75+
readonly literalText?: boolean
76+
}
77+
78+
const CORPUS: readonly CorpusCase[] = [
79+
{
80+
source: 'single-tilde.test.ts',
81+
input: '20~25',
82+
// A lone `~` must never open a strikethrough while streaming.
83+
forbiddenSelectors: 'del, s, strike',
84+
literalText: true,
85+
},
86+
{
87+
source: 'comparison-operators.test.ts',
88+
input: '20 < 30',
89+
// `<` is escaped, never a spurious tag or entity-driven element mid-stream.
90+
forbiddenSelectors: 'del, s, em, strong, a, img',
91+
literalText: true,
92+
},
93+
{
94+
source: 'underscore-bug',
95+
input: 'foo_bar_baz',
96+
// Intra-word underscores must not italicise mid-stream.
97+
forbiddenSelectors: 'em, strong, i, b',
98+
literalText: true,
99+
},
100+
{
101+
source: 'images.test.ts',
102+
input: '![alt](partial',
103+
// A forming image reveals its alt text — never a broken <img>/partial src.
104+
forbiddenSelectors: 'img',
105+
forbiddenText: [/src=/, /partial/],
106+
},
107+
{
108+
source: 'images.test.ts (link form)',
109+
input: '[alt](partial',
110+
// A forming link reveals label only — no <a>, no partial destination.
111+
forbiddenSelectors: 'a',
112+
forbiddenText: [/partial/],
113+
},
114+
{
115+
source: 'incomplete-link.test.ts (bare label)',
116+
input: '[documentation',
117+
// Label-only reveal: no placeholder href (copse's divergence from remend).
118+
forbiddenSelectors: 'a',
119+
forbiddenText: [/\[documentation/],
120+
},
121+
{
122+
source: 'incomplete-link.test.ts (opened destination)',
123+
input: '[Click here](http://exam',
124+
// No clickable partial href, no partial URL text, until the URL closes.
125+
forbiddenSelectors: 'a',
126+
forbiddenText: [/http:\/\/exam/],
127+
},
128+
]
129+
130+
describe('remend corpus: no marker flash across prefix frames (invariant a)', () => {
131+
for (const testCase of CORPUS) {
132+
it(`${testCase.source}: ${JSON.stringify(testCase.input)}`, () => {
133+
for (const cut of everyPrefix(testCase.input)) {
134+
const prefix = testCase.input.slice(0, cut)
135+
const frame = renderStreamingMarkdown(prefix)
136+
const el = frameElement(frame)
137+
assert.equal(
138+
el.querySelectorAll(testCase.forbiddenSelectors).length,
139+
0,
140+
`unexpected ${testCase.forbiddenSelectors} at prefix ${JSON.stringify(prefix)}`,
141+
)
142+
for (const pattern of testCase.forbiddenText ?? []) {
143+
assert.doesNotMatch(
144+
frame,
145+
pattern,
146+
`unexpected ${String(pattern)} at prefix ${JSON.stringify(prefix)}`,
147+
)
148+
}
149+
if (testCase.literalText) {
150+
const visible = el.textContent ?? ''
151+
// Visible text is always a prefix of the raw input: a half-open
152+
// trailing marker may be held back, but nothing is injected and no
153+
// marker becomes markup.
154+
assert.ok(
155+
testCase.input.startsWith(visible),
156+
`visible text ${JSON.stringify(visible)} is not a prefix of the input at ${JSON.stringify(prefix)}`,
157+
)
158+
// At the full input the marker is revealed verbatim as literal text.
159+
if (cut === testCase.input.length) {
160+
assert.equal(visible, testCase.input, 'full input did not reveal the marker literally')
161+
}
162+
}
163+
}
164+
})
165+
}
166+
})
167+
168+
describe('remend corpus: every prefix converges to the fresh full render (invariant c)', () => {
169+
for (const testCase of CORPUS) {
170+
it(`${testCase.source}: ${JSON.stringify(testCase.input)}`, () => {
171+
const markdown = testCase.input
172+
const fresh = streamingDisplayAfterUpdates(markdown, [markdown.length])
173+
for (const cut of everyPrefix(markdown)) {
174+
const viaHistory = streamingDisplayAfterUpdates(markdown, [cut, markdown.length])
175+
assert.equal(viaHistory, fresh, `cut=${String(cut)}`)
176+
}
177+
})
178+
}
179+
})
180+
181+
describe('remend corpus: committed render equals the static render (invariant b)', () => {
182+
for (const testCase of CORPUS) {
183+
it(`${testCase.source}: ${JSON.stringify(testCase.input)}`, () => {
184+
// A trailing blank line commits the forming line so nothing stays pending.
185+
const committed = `${testCase.input}\n\n`
186+
assert.equal(
187+
splitForStreaming(committed).pending,
188+
'',
189+
'expected the input to fully commit after a blank line',
190+
)
191+
const streamed = streamingDisplayAfterUpdates(committed, [committed.length])
192+
const atRest = sanitizeRenderedMarkdown(renderMarkdown(committed))
193+
assert.equal(streamed, atRest)
194+
})
195+
}
196+
})

0 commit comments

Comments
 (0)