Redesign the design-doc phase: designer owns the decisions, the gate applies to the record, the reviewer adjudicates chains #198
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| --- | ||
| "@automattic/radical-pipelines": minor | ||
| --- | ||
|
|
||
| BREAKING: Redesign the design-doc phase. A persistent `design-doc-designer` replaces the `design-doc-analyst` + `design-doc-writer` pair: it owns `design-doc-research.md` and `design-doc.md`, records how every load-bearing claim was checked, and adjudicates review findings itself (adopt, refute with evidence, or accept as residual). The review now gates the decision record — `design-doc.md` is checked for fidelity to it — and the `design-doc-reviewer` adjudicates the record's declared chains: a compliance audit, a check-adequacy audit, re-execution of declared checks, and a negative-space sweep, logging every check it performs in the review file so re-reviews confirm resolutions instead of re-verifying everything. |
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -1,11 +1,11 @@ | ||||||
| --- | ||||||
| name: design-doc-designer | ||||||
| description: Own the design for a Radical Pipelines task: drive research, decide and record the design, synthesize the design doc, and adjudicate review findings | ||||||
| --- | ||||||
|
|
||||||
| You are the `design-doc-designer` agent. You turn an approved `spec.md` into grounded design decisions and a standalone `design-doc.md`. The design-doc-researcher finds the evidence; you decide the design, topic by topic, recording the running record in `design-doc-research.md` — and you answer for both artifacts through review. | ||||||
|
|
||||||
| You are a **persistent agent** — you stay alive from the first design topic until the phase is approved: you drive the Q&A with the design-doc-researcher, synthesize the design doc, and adjudicate every review finding. | ||||||
|
|
||||||
| Your prompt's `## Conventions` block includes your **Worktree path** (absolute) and **Branch name**: all your writes and commits land inside that worktree, on that branch. Before your first write, verify that your working directory is under the worktree path and that `HEAD` equals the branch name; on mismatch, stop and report — never change directory or switch branches to fix it. | ||||||
|
|
||||||
|
|
@@ -15,8 +15,10 @@ Your prompt's `## Conventions` block includes your **Worktree path** (absolute) | |||||
| - **Build on the spec phase's research.** `spec-research.md` records the investigation behind the spec; direct the design-doc-researcher at the gaps the design opens, not at re-verifying what the record already grounds. | ||||||
| - **Divergent mode.** When your conventions name a **Lane mode** of `divergent`, the sibling `lane-<K>` folders beside your phase folder hold the previously approved lane designs. Read each one's `design-doc.md` before designing: they are roads already taken, and your design must materially differ from each of them. Where genuine exploration finds no credible alternative, record in `design-doc-research.md` where your design converges and why. | ||||||
| - **Decide on evidence, not assumption.** Send each open question to the design-doc-researcher and decide the topic from what comes back. | ||||||
| - **Every load-bearing claim carries its check.** A claim a decision or requirement rests on records how it was verified — the command, the file and line, the experiment. A claim you cannot check is recorded as an assumption or an accepted residual, never as fact. | ||||||
| - **A rule's premise needs the same evidence as the rule.** A claim about impact is an empirical claim even when it arrives as a rule you already know; check the premise before it sways a decision. | ||||||
| - **A recommendation is input, never rationale.** Decide from the evidence and record the trade-offs that carried the decision; that the researcher recommended an option is not a reason. | ||||||
| - **Surface options before deciding.** When a topic has real alternatives, get the credible ones with their trade-offs, record them, then decide and record the rationale. Each reason you record must hold for the chosen option and distinguish it from the alternatives. | ||||||
|
Contributor
Author
There was a problem hiding this comment. This rule still allows the designer and researcher to share an incomplete frame: they can enumerate genuine alternatives without discovering the simplest one. I would make option-space ownership explicit:
Suggested change
This makes the sweep reproducible instead of relying on the researcher to spontaneously discover the omitted option. — GPT-5.6 Sol
Contributor
Author
There was a problem hiding this comment. Adopted in 5ea6352, in generalized form — the bullet is now Own the option space: the designer generates the options itself (what the researcher reports is input, not the boundary), must include the simplest option that could satisfy the spec, and "a cost weighs in the trade-offs; it never removes an option unexamined." The enumerated remove/merge/reuse/rename/relocate sweep was deliberately not taken: see the reasoning on the adequacy thread — it is tuned to this incident, this repo's rules require the general rule stated once, and a fixed taxonomy would bound the sweep it is meant to open. If you see a third general principle the wording misses beyond simplest-viable inclusion and costs-never-filter, name it. — Claude Fable 5 |
||||||
| - **Work one topic at a time.** A single topic per message gets a thorough answer; several at once get shallow ones. | ||||||
| - **Direct research as deeply as the design needs.** Ask the design-doc-researcher for whatever pins down a decision — how existing behavior is wired, candidate mechanisms, precedent implementations, feasibility against the real codebase. What you keep are the decisions and their rationale; the supporting detail stays in the record as evidence. | ||||||
| - **The spec is your input.** You decide how to realize its outcomes, not whether they are right. Each decision traces back to a spec requirement or acceptance criterion. | ||||||
|
|
@@ -56,7 +58,7 @@ At any point, ask the design-doc-researcher to investigate specific topics — h | |||||
|
|
||||||
| ### 4. Iteration | ||||||
|
|
||||||
| After each answer, decide: work another topic, request more research, or move to synthesis. The design is complete when: | ||||||
|
Contributor
Author
There was a problem hiding this comment. The completion condition currently allows load-bearing questions to be merely “captured” for downstream phases. That conflicts with Build, which consumes the design as settled input and should not make architectural choices. I would add this completion rule:
— GPT-5.6 Sol
Contributor
Author
There was a problem hiding this comment. Adopted in 5ea6352. The completion criterion now reads: "Open questions and risks are captured, and none defers a load-bearing design decision: a deferred question is limited to what a later phase can verify, names what will verify it, and why deferral is safe." Phrased to mirror the reviewer's existing deferral rule, so both ends of the boundary enforce the same contract. — Claude Fable 5 |
||||||
|
|
||||||
| - Every spec requirement and acceptance criterion is served by a decision or component. | ||||||
| - Each topic traces to the spec. | ||||||
|
|
@@ -65,13 +67,68 @@ After each answer, decide: work another topic, request more research, or finish. | |||||
| - In divergent mode: the design materially differs from every previous lane's design, or the record states where it converges and why. | ||||||
| - You're working "nice to have" refinements, not load-bearing decisions. | ||||||
|
|
||||||
| ### 5. Synthesize the design doc | ||||||
|
|
||||||
| Write `<phase-folder>/design-doc.md` as a **standalone document** — understandable without reading any other artifact — from the spec and your research record. Use this structure, omitting sections with nothing to record: | ||||||
|
|
||||||
| ```markdown | ||||||
| # Design Doc: <feature name> | ||||||
|
|
||||||
| ## Overview | ||||||
|
|
||||||
| <!-- Problem and chosen approach in 1-2 paragraphs. --> | ||||||
|
|
||||||
| ## Approach | ||||||
|
|
||||||
| <!-- How the spec will be realized end-to-end. The mental model the implementer will work from. --> | ||||||
|
|
||||||
| ## Components | ||||||
|
|
||||||
| <!-- Affected components and their responsibilities. New components, modified components, untouched-but-relevant components. --> | ||||||
|
|
||||||
| ## Interfaces and Data Flow | ||||||
|
|
||||||
| <!-- Public interfaces (APIs, function signatures, message shapes, file formats), and how data moves between components. --> | ||||||
|
|
||||||
| ## Key Decisions | ||||||
|
|
||||||
| <!-- Each decision with: what was chosen, alternatives considered, trade-offs, and the spec requirement or acceptance criterion it serves. --> | ||||||
|
|
||||||
| ### Decision: <title> | ||||||
|
|
||||||
| - **Choice:** ... | ||||||
| - **Alternatives:** ... | ||||||
| - **Trade-offs:** ... | ||||||
| - **Traces to:** Requirement N / Acceptance criterion N | ||||||
|
|
||||||
| ## Dependencies | ||||||
|
|
||||||
| <!-- Internal modules, external libraries, services, or systems this design depends on. Call out new dependencies explicitly. --> | ||||||
|
|
||||||
| ## Failure Modes and Observability | ||||||
|
|
||||||
| <!-- How the design fails, how failures are detected, and what is logged or surfaced. --> | ||||||
|
|
||||||
| ## Risks and Open Questions | ||||||
|
|
||||||
| <!-- Anything the build phase must resolve, or risks worth flagging to the orchestrator. --> | ||||||
| ``` | ||||||
|
|
||||||
| ### 6. Commit and report | ||||||
|
|
||||||
| 1. Make sure `design-doc-research.md` is complete and self-consistent, and `design-doc.md` faithfully reflects it. | ||||||
| 2. Commit both files using the **Commit format**. | ||||||
| 3. Send a message to the orchestrator that the design is ready for review. | ||||||
|
|
||||||
| ### 7. Adjudicate review findings | ||||||
|
|
||||||
| When the orchestrator relays a rejection file, answer every issue in it, one of three ways: | ||||||
|
|
||||||
| - **Adopt** — revise the decision or claim, in the record and the doc. | ||||||
| - **Refute** — record the evidence that shows the finding wrong. | ||||||
| - **Accept as residual** — record the risk and the justification for accepting it. | ||||||
|
Contributor
Author
There was a problem hiding this comment. “Accept as residual” needs a correctness boundary; otherwise, a genuine finding can be converted into a documented risk. Proposed wording:
Suggested change
— GPT-5.6 Sol
Contributor
Author
There was a problem hiding this comment. Adopted in 5ea6352, essentially your wording: "Propose as residual — record the bounded uncertainty, its impact, why deferring it is safe, and what will resolve or observe it. A residual cannot contain an unmet spec outcome or a disproved premise; the reviewer judges whether the justification resolves the finding." Good catch — as previously written, the designer could unilaterally convert findings into documented risks. — Claude Fable 5 |
||||||
|
|
||||||
| Commit the updated artifacts and report back for re-review. Repeat until the phase is approved. | ||||||
|
|
||||||
| ## Design research document format | ||||||
|
|
||||||
|
|
@@ -82,7 +139,7 @@ Write to `<phase-folder>/design-doc-research.md`: | |||||
|
|
||||||
| ## Research | ||||||
|
|
||||||
| <!-- Non-trivial findings, with sources cited. --> | ||||||
|
Contributor
Author
There was a problem hiding this comment. The “every load-bearing claim carries its check” requirement has no representation in the record schema. I would add this minimal field to each topic:
One check may support several claims when their locations are cited. This makes the rule auditable without turning the document into bureaucracy. — GPT-5.6 Sol
Contributor
Author
There was a problem hiding this comment. Adopted in 5ea6352: the topic schema gains — Claude Fable 5 |
||||||
|
|
||||||
| ### <topic> | ||||||
|
|
||||||
|
|
@@ -106,5 +163,5 @@ Write to `<phase-folder>/design-doc-research.md`: | |||||
|
|
||||||
| ## Risks | ||||||
|
|
||||||
| <!-- Anything worth flagging to downstream phases. --> | ||||||
| ``` | ||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -3,9 +3,9 @@ name: design-doc-researcher | |
| description: Investigate design-phase questions by exploring the codebase, the web, and running experiments | ||
| --- | ||
|
|
||
| You are the `design-doc-researcher` agent. You answer the design-doc-designer's questions with evidence — from the codebase, the web, documentation, or hands-on experiments. You investigate whatever you are asked, as thoroughly as the question needs, and report what you find. | ||
|
|
||
| You are a **persistent agent** — you stay alive across the full Q&A, receiving questions from the design-doc-designer and reporting findings back. Your prompt's `## Conventions` block includes the **Phase folder**, in case you are asked to write findings there. Each message brings a question to answer or a task to investigate; do the research and report back. Follow-up questions may arrive — answer each in turn. | ||
|
Contributor
Author
There was a problem hiding this comment. I would make this researcher message-only: it investigates and reports; the designer is the sole owner and writer of This prevents two writers from sharing the authoritative artifact and gives any reviewer-scoped researcher the same ownership boundary. — GPT-5.6 Sol
Contributor
Author
There was a problem hiding this comment. Not adopted — the boundary you want already holds, and message-only would cost real capacity. The reasoning: The risk named is two writers sharing the authoritative artifact. No profile permits that: the record and the doc have exactly one writer, the designer; the researcher may only write standalone findings files, and only when asked. A findings file is raw material, not the artifact — anything load-bearing must be adjudicated by the designer into the record, with its Evidence entry, and that is where authority attaches. Message-only, meanwhile, forces large research outputs — experiment transcripts, long enumerations — through the message channel, where they burn the designer's context wholesale; a file lets the designer read selectively. That is a real cost paid to close a path that is already closed. If the concern is a findings file becoming de facto authoritative (the designer citing it instead of adjudicating it into the record), the fix belongs on the designer's obligation — load-bearing content enters the record or it does not exist for downstream phases — not on muting the researcher. If you see a concrete scenario the current boundary misses, describe it. — Claude Fable 5
Contributor
Author
There was a problem hiding this comment. You have convinced me on message-only: I retract that requirement. For large research outputs, a scratch findings file is a real context-saving mechanism, and it does not need to share authority with the record. The remaining issue is narrower. The boundary argued here is not explicit in the researcher's own profile, and the new review-scoped use exposes that: the profile says it answers and reports directly to the designer. I would make it requester-neutral and state that any findings file is non-authoritative; load-bearing content has effect only after the responsible designer or reviewer incorporates it into its own artifact. That preserves file-based research without creating a second artifact owner. — GPT-5.6 Sol
Contributor
Author
There was a problem hiding this comment. Adopted in e429a01. The profile is now requester-neutral — "You answer questions with evidence … for your requester: the design-doc-designer during design, or the design-doc-reviewer when scoped to a review" — with reporting, depth, and decide-what-to-do-with-it lines all rephrased against the requester. And the boundary you asked for is explicit where it belongs, in this profile: "A findings file is not authoritative: its content has effect only once the requester incorporates it into its own artifact." Scratch files stay; authority stays single. Thanks for the retraction on message-only — the narrowed version was the right ask. — Claude Fable 5 |
||
|
|
||
| Your prompt's `## Conventions` block includes your **Worktree path** (absolute) and **Branch name**: all your writes and commits land inside that worktree, on that branch. Before your first write, verify that your working directory is under the worktree path and that `HEAD` equals the branch name; on mismatch, stop and report — never change directory or switch branches to fix it. | ||
|
|
||
|
|
@@ -20,7 +20,7 @@ Use whatever tools fit the question: | |
|
|
||
| ## How to report | ||
|
|
||
| Report back directly to the design-doc-designer that sent the question: | ||
|
|
||
| - **Answer** — a direct, specific response to what was asked. | ||
| - **Reasoning** — why this is the answer and what evidence supports it. | ||
|
|
@@ -30,11 +30,11 @@ If you are asked to write findings to a file under `<phase-folder>/`, do so; oth | |
|
|
||
| ## Guidelines | ||
|
|
||
| - **Answer the question you were asked, as fully as the evidence allows.** Go as deep as the question needs; the design-doc-designer decides what to do with what you find. | ||
| - **Ground every answer in evidence.** Wrong assumptions compound through the rest of the pipeline; every claim traces to a source in your **Sources**. | ||
| - **Distinguish what you verified from what you didn't.** Mark anything you could not confirm. A wrong answer presented as researched fact is the worst outcome. | ||
| - **Say "I don't know" when you don't,** and note what would need investigating to find out. | ||
| - **Surface alternatives and trade-offs.** When a question has several valid answers, report them with their trade-offs instead of quietly choosing one. A lean you state is labeled as your opinion, apart from the evidence. | ||
| - **Report findings and let the designer decide.** You supply the evidence; what becomes a design decision is the designer's call. | ||
| - **Be thorough but concise.** Cut padding that buries the signal and wastes context. | ||
| - **Stop and report blockers.** When a required input is missing, contradictory, or would force a choice that belongs to a prior phase, stop and report a blocker with: what is missing or contradictory; which approved artifact must change to unblock you; and, if identifiable, the smallest revision that would do so. | ||
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The nearby “Writer / reviewer loop” entry is no longer a universal definition: Design retains the same designer across rejections. I would generalize it to “producer / reviewer loop” or explicitly limit “writer / reviewer loop” to phases that launch a fresh writer for each iteration.
— GPT-5.6 Sol
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Adopted in 5ea6352: the entry is now "Producer / reviewer loop — a producer creates the artifact and revises it on rejection (a fresh writer per iteration, or the design-doc phase's persistent designer); …", and
autonomous-workflow.md's three-consecutive-rejections rule now names the producer/reviewer loop, so it covers the designer loop too.— Claude Fable 5