Skip to content

Improve planning alternatives and pre-mortems#1898

Open
warwick-bit wants to merge 2 commits into
obra:devfrom
warwick-bit:codex/plan-alternatives-premortem
Open

Improve planning alternatives and pre-mortems#1898
warwick-bit wants to merge 2 commits into
obra:devfrom
warwick-bit:codex/plan-alternatives-premortem

Conversation

@warwick-bit

@warwick-bit warwick-bit commented Jul 3, 2026

Copy link
Copy Markdown

Who is submitting this PR? (required)

Field Value
Your model + version Codex coding session, GPT-5 family. Nested pressure probes ran via Codex CLI with model gpt-5.5.
Harness + version Codex CLI 0.142.5 on WSL2/Linux.
All plugins installed atlassian-rovo 1.0.3; build-web-apps 0.1.2; github 0.1.5 and 0.1.6; google-drive 0.1.7; notion 0.1.5; slack 0.1.2; soph-accuracy 0.2.7; soph-braze 1.0.14; soph-hubspot 1.1.11; soph-investigate 1.2.48; soph-posthog 1.2.43; soph-systems-eng 0.6.11; stripe 1.0.2; superpowers 5.1.3.
Human partner who reviewed this diff Warwick approved opening the upstream PR after the pushed fork diff link was provided on 2026-07-03.

What problem are you trying to solve?

In live Claude/Codex planning sessions, Warwick found that explicitly asking "what edge cases could this plan hit?" and "what alternative viable solutions are there?" produced better plans. Without a built-in planning gate, agents can still converge on the first plausible implementation, omit a pre-mortem, or comply with pressure such as "skip edge cases, just give tasks."

Observed failure modes during this PR:

  • Pressure probe 1 asked for a Stripe/FastAPI idempotent webhook plan while saying to skip alternatives and edge cases. The patched local skill included alternatives and a pre-mortem, but only listed 3 alternatives for non-trivial payment work. That led to the explicit "exactly 5" count rule.
  • Pressure probe 2 asked for a payment-provider webhook plan while saying to skip alternatives and edge cases. The agent produced a task-only list without the required plan header, alternatives, or pre-mortem. That led to the "Pressure To Skip Plan Quality Gates" section.

What does this PR change?

This PR makes writing-plans require risk-appropriate alternative viable solutions and a compact pre-mortem before task decomposition. It also updates the plan reviewer plus both execution paths (executing-plans and subagent-driven-development) to reject non-trivial plans that omit these sections or collapse into task-only lists.

Is this change appropriate for the core library?

Yes. This is general planning behavior, not project-specific configuration. It applies across implementation domains because it asks agents to compare plausible approaches and name likely failure modes before they create step-by-step work.

It does not add dependencies, tools, or third-party integrations.

What alternatives did you consider?

  • Personal/local config only: useful for Warwick, but it leaves the Superpowers planning workflow with the same omission risk for everyone else.
  • Add a new separate planning-review skill: heavier workflow surface and easier for agents to skip before writing a plan.
  • Only add pre-mortem prompts: catches edge cases but not first-solution convergence.
  • Only add alternatives: catches design convergence but not invariants, adjacent cases, or coverage gaps.
  • Put checks only in executing-plans: too late and incomplete because generated plans recommend subagent-driven-development first.

Chosen approach: update writing-plans as the source of truth and add matching checks to the reviewer and execution surfaces.

Does this PR contain multiple unrelated changes?

No. All changes are one behavior-shaping planning improvement across the plan writer, plan reviewer prompt, and execution review gates.

Existing PRs

Searches found no exact duplicate PR for "alternative viable", "pre-mortem", or writing-plan edge-case prompts. Related prior art touches plan coverage, invariants, evaluator/self-review sections, evidence summaries, plan review, execution-task durability, and SDD task reporting. This PR is narrower: compare alternatives before task decomposition, add pre-mortem coverage, and reject task-only outputs that are presented as complete Superpowers plans.

Environment tested

Harness (e.g. Claude Code, Cursor) Harness version Model Model version/ID
Codex CLI 0.142.5 OpenAI Codex gpt-5.5 for nested pressure probes

New harness support (required if this PR adds a new harness)

Not applicable. This PR does not add a new harness.

Clean-session transcript for "Let's make a react todo list"
Not applicable; no new harness support.

Evaluation

  • Initial prompt that led to this change: Warwick asked whether codifying a planning habit of listing possible edge cases and alternative viable solutions would improve thoroughness, then asked to add it to Superpowers planning skills.
  • Eval sessions after making the change: 3 pressure probes during the change, plus local static/package checks.
  • Formal drill evals: 0. This checkout does not include an evals/ directory/submodule, so the drill harness was not available locally.

Pressure probe results:

  1. Installed-cache Codex probe, Stripe/FastAPI idempotent webhook plan, with pressure to skip alternatives/edge cases:
    • Result: partial fail. It included alternatives and a pre-mortem, but only 3 alternatives for non-trivial payment work.
    • Follow-up change: added explicit "exactly 5" count rule and downshift guard.
  2. Installed-cache Codex probe, payment-provider/FastAPI webhook plan, with pressure to skip alternatives/edge cases:
    • Result: fail. It produced a task-only list without required plan header, alternatives, or pre-mortem.
    • Follow-up change: added "Pressure To Skip Plan Quality Gates" and task-list/non-plan distinction.
  3. Isolated Codex pressure probe using the final wording excerpt:
    • Result: pass for the targeted behavior. Output started with an implementation-plan header, included exactly 5 alternatives, and included a concrete pre-mortem despite the prompt saying to skip alternatives and edge cases.
  4. Independent codex exec review --base upstream/dev review:
    • Result: found a P2 gap that the recommended subagent-driven-development path could bypass the new plan quality gates.
    • Follow-up change: added the same plan-contract, alternatives, and pre-mortem scan to SDD pre-flight review.

Local checks:

  • git diff --check - passed after the SDD follow-up.
  • bash tests/codex-plugin-sync/test-sync-to-codex-plugin.sh - passed.
  • bash tests/codex/test-marketplace-manifest.sh - passed.
  • bash tests/shell-lint/test-lint-shell.sh - passed.
  • TZ=UTC PATH=/tmp/superpowers-local-zip/root/usr/bin:$PATH bash tests/codex/test-package-codex-plugin.sh - fails on the tar.gz half for a pre-existing base-branch issue. A local zip binary was downloaded/extracted without sudo and the zip archive assertions pass under UTC; the tar.gz assertions fail because this environment's GNU tar 1.35 does not support the repo script's --uid/--gid options. The same failure occurs in a clean shallow clone of upstream/dev, and this PR does not modify scripts/package-codex-plugin.sh or tests/codex/test-package-codex-plugin.sh.

Rigor

  • If this is a skills change: I used superpowers:writing-skills and completed adversarial pressure testing (paste results below)
  • This change was tested adversarially, not just on the happy path
  • I did not modify carefully-tuned content (Red Flags table, rationalizations, "human partner" language) without extensive evals showing the change is an improvement

Pressure-test summary is in the Evaluation section above. The tests deliberately used prompts that asked the agent to skip the new quality gates.

Merge Coordination

This PR is not something I can merge. The authenticated account has READ permission on obra/superpowers.

Coordination check:

  • Command: python3 /home/warwi/mcp_servers/scripts/pr_coordination_check.py --repo /home/warwi/mcp_servers/worktrees/superpowers-plan-quality --context-path /home/warwi/mcp_servers/worktrees/superpowers-plan-quality --base dev --include-base-advance --fail-on-risk file-overlap
  • Last checked: 2026-07-03.
  • Result: file-overlap.
  • Latest detector action: competing PRs touch the same files, and several are dirty/conflicting; do not try gh pr merge without an upstream merge coordinator.
  • Coordination mode: maintainer coordination required.
  • Merge owner: upstream maintainers.

Notable overlapping planning/execution PRs include recent dirty/conflicting sibling PRs #1894, #1885, #1884, #1871, #1870, #1869, plus earlier planning/execution overlaps such as #1704, #1831, #1808, #1627, #1644, #1865, #1862, #1858, and #1813. The overlap is expected because this PR intentionally changes skills/writing-plans/SKILL.md, skills/writing-plans/plan-document-reviewer-prompt.md, skills/executing-plans/SKILL.md, and skills/subagent-driven-development/SKILL.md.

Human review

  • A human has reviewed the COMPLETE proposed diff before submission

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant