From b106106b6fe6251da2bdccd55e3dcaf9afe7faf8 Mon Sep 17 00:00:00 2001 From: vergil_wong Date: Wed, 22 Jul 2026 16:57:24 +0800 Subject: [PATCH] feat: add behavior-preserving optimization reference --- .agents/skills/autoresearch/SKILL.md | 5 ++ .agents/skills/autoresearch/autoresearch.md | 9 +++ .agents/skills/autoresearch/plan.md | 6 ++ .../behavior-preserving-optimization.md | 61 +++++++++++++++++++ .claude/commands/autoresearch.md | 9 +++ .claude/commands/autoresearch/plan.md | 6 ++ .claude/skills/autoresearch/SKILL.md | 5 ++ .../behavior-preserving-optimization.md | 61 +++++++++++++++++++ .opencode/commands/autoresearch.md | 9 +++ .opencode/commands/autoresearch_plan.md | 6 ++ .opencode/skills/autoresearch/SKILL.md | 5 ++ .../behavior-preserving-optimization.md | 61 +++++++++++++++++++ AGENTS.md | 2 +- .../autoresearch/skills/autoresearch/SKILL.md | 5 ++ .../skills/autoresearch/autoresearch.md | 9 +++ .../autoresearch/skills/autoresearch/plan.md | 6 ++ .../behavior-preserving-optimization.md | 61 +++++++++++++++++++ 17 files changed, 325 insertions(+), 1 deletion(-) create mode 100644 .agents/skills/autoresearch/references/behavior-preserving-optimization.md create mode 100644 .claude/skills/autoresearch/references/behavior-preserving-optimization.md create mode 100644 .opencode/skills/autoresearch/references/behavior-preserving-optimization.md create mode 100644 plugins/autoresearch/skills/autoresearch/references/behavior-preserving-optimization.md diff --git a/.agents/skills/autoresearch/SKILL.md b/.agents/skills/autoresearch/SKILL.md index d8e4841..eeec57b 100644 --- a/.agents/skills/autoresearch/SKILL.md +++ b/.agents/skills/autoresearch/SKILL.md @@ -64,6 +64,11 @@ Print a banner on every invocation: `[autoresearch] mode: classic | orchestrator Activated when a plain-language goal is given without `Metric:`/`Verify:`. Classifies the goal into a **Goal archetype** — see `references/orchestrator-routing.md` for the archetype table and router decision table. +For performance, latency, throughput, runtime, CPU, memory, allocations, binary +size, power, energy, bandwidth, or compute-cost goals, read +`references/behavior-preserving-optimization.md`. The working metric ranks +candidates; independent behavior evidence decides whether a candidate survives. + **Two modes based on archetype:** - **Orchestration loop** — predicate-bearing archetypes (ship-ready, optimize-metric, fix-broken, harden, build-feature, explore). Goal has a mechanical Success predicate; the loop runs until that predicate is met. - **Single-pass dispatch** — subjective/terminal archetypes (document, what-to-build, decide-design). Routes once to the fitting subcommand (learn / improve / reason), lets it self-terminate, then reports. No loop, no Plateau, no ship gate. diff --git a/.agents/skills/autoresearch/autoresearch.md b/.agents/skills/autoresearch/autoresearch.md index 7eb1cf1..aad1df8 100644 --- a/.agents/skills/autoresearch/autoresearch.md +++ b/.agents/skills/autoresearch/autoresearch.md @@ -29,6 +29,15 @@ If Goal, Scope, Metric, or Verify missing → use request_user_input (single bat Q4 (Guard): "Safety command that must always pass?" — options: test cmd, build cmd, skip If ALL provided inline → skip setup, proceed directly. +## Conditional Performance Reference + +If Goal or Metric targets performance, latency, throughput, runtime, CPU, +memory, allocations, binary size, power, energy, bandwidth, or compute cost, read +`.claude/skills$autoresearch/references/behavior-preserving-optimization.md` +before establishing the baseline. Apply its exploration/confirmation split: a +metric-improving `keep` is an exploratory incumbent, not proof that behavior is +preserved. + ## Precondition Checks 1. Verify git repo exists (`git rev-parse --git-dir`) diff --git a/.agents/skills/autoresearch/plan.md b/.agents/skills/autoresearch/plan.md index eed5438..de785da 100644 --- a/.agents/skills/autoresearch/plan.md +++ b/.agents/skills/autoresearch/plan.md @@ -43,6 +43,12 @@ For metric-driven goals: - Determine direction: higher_is_better or lower_is_better - Propose metric name and description +For performance, latency, throughput, runtime, CPU, memory, allocations, binary +size, power, energy, bandwidth, or compute-cost goals, read +`.claude/skills$autoresearch/references/behavior-preserving-optimization.md`. +Derive a behavior/resource Guard and a held-out confirmation plan separately +from the headline metric; do not let the optimized metric certify itself. + For subjective goals: - Suggest proxy metrics where possible - Or recommend $autoresearch reason for non-measurable goals diff --git a/.agents/skills/autoresearch/references/behavior-preserving-optimization.md b/.agents/skills/autoresearch/references/behavior-preserving-optimization.md new file mode 100644 index 0000000..d3271cf --- /dev/null +++ b/.agents/skills/autoresearch/references/behavior-preserving-optimization.md @@ -0,0 +1,61 @@ +# Behavior-Preserving Performance Optimization + +Load this reference when the objective is performance, latency, throughput, +runtime, CPU, memory, allocations, binary size, power, energy, bandwidth, or +compute cost and the system must keep doing the same useful job. + +## Separate the Search Signal from Acceptance + +The metric being optimized cannot also prove behavior preservation. A candidate +can become faster by skipping work, narrowing inputs, lowering quality, moving +cost elsewhere, or changing timing outside the benchmark's view. + +Before the baseline, define: + +- the performance objective, workload, measurement method, and smallest useful + gain; +- user-visible, protocol, perceptual, or numerical behavior that must remain; +- independent behavior checks that reject at least one known-bad change; +- resource boundaries for memory, startup, energy, bandwidth, deadline misses, + and tail latency where relevant; +- comparable artifact, configuration, hardware, and input provenance; +- representative inputs for search and separate held-out or adversarial inputs + for confirmation; +- a rollback point. + +If a guard was chosen mainly because it is easy to keep green, replace it. + +## Run Two Loops + +Use the normal autoresearch loop for exploration. Reversible batches may locate +promising directions, and the working metric may rank them. A `keep` decision +means only "better exploratory incumbent"; it does not establish causality or +readiness. + +Confirm each promising direction before calling it behavior-preserving: + +1. Rebuild it as an isolated, reviewable change. +2. Re-run the objective and independent behavior checks on representative plus + held-out or adversarial workloads. +3. Compare end-to-end and tail costs, not only the edited function or average. +4. Reject the candidate when any behavior or resource boundary fails. +5. Preserve raw baseline/candidate evidence and comparable provenance. +6. Record unmeasured behavior and the rollback path. + +## Resist Proxy Success + +A green metric or guard proves only that the declared observations passed. It +does not prove that the observations represent the user's real objective, that +the measurements are truthful, or that no important behavior is missing. + +Before final acceptance, a reviewer should be able to identify the causal +change, inspect the raw evidence, see an independent check reject a known-bad +case, and explain what remains unmeasured. + +Reject these arguments: + +- "The benchmark is much faster, so the change is good." +- "The outputs look close enough." +- "The guard passed, so behavior is preserved." +- "We can isolate the five tweaks after they land." +- "The benchmark input is representative by definition." diff --git a/.claude/commands/autoresearch.md b/.claude/commands/autoresearch.md index 65882f1..b7d6424 100644 --- a/.claude/commands/autoresearch.md +++ b/.claude/commands/autoresearch.md @@ -29,6 +29,15 @@ If Goal, Scope, Metric, or Verify missing → use AskUserQuestion (single batche Q4 (Guard): "Safety command that must always pass?" — options: test cmd, build cmd, skip If ALL provided inline → skip setup, proceed directly. +## Conditional Performance Reference + +If Goal or Metric targets performance, latency, throughput, runtime, CPU, +memory, allocations, binary size, power, energy, bandwidth, or compute cost, read +`.claude/skills/autoresearch/references/behavior-preserving-optimization.md` +before establishing the baseline. Apply its exploration/confirmation split: a +metric-improving `keep` is an exploratory incumbent, not proof that behavior is +preserved. + ## Precondition Checks 1. Verify git repo exists (`git rev-parse --git-dir`) diff --git a/.claude/commands/autoresearch/plan.md b/.claude/commands/autoresearch/plan.md index b5afe12..7f3605d 100644 --- a/.claude/commands/autoresearch/plan.md +++ b/.claude/commands/autoresearch/plan.md @@ -43,6 +43,12 @@ For metric-driven goals: - Determine direction: higher_is_better or lower_is_better - Propose metric name and description +For performance, latency, throughput, runtime, CPU, memory, allocations, binary +size, power, energy, bandwidth, or compute-cost goals, read +`.claude/skills/autoresearch/references/behavior-preserving-optimization.md`. +Derive a behavior/resource Guard and a held-out confirmation plan separately +from the headline metric; do not let the optimized metric certify itself. + For subjective goals: - Suggest proxy metrics where possible - Or recommend /autoresearch:reason for non-measurable goals diff --git a/.claude/skills/autoresearch/SKILL.md b/.claude/skills/autoresearch/SKILL.md index 45969d2..3001a29 100644 --- a/.claude/skills/autoresearch/SKILL.md +++ b/.claude/skills/autoresearch/SKILL.md @@ -64,6 +64,11 @@ Print a banner on every invocation: `[autoresearch] mode: classic | orchestrator Activated when a plain-language goal is given without `Metric:`/`Verify:`. Classifies the goal into a **Goal archetype** — see `references/orchestrator-routing.md` for the archetype table and router decision table. +For performance, latency, throughput, runtime, CPU, memory, allocations, binary +size, power, energy, bandwidth, or compute-cost goals, read +`references/behavior-preserving-optimization.md`. The working metric ranks +candidates; independent behavior evidence decides whether a candidate survives. + **Two modes based on archetype:** - **Orchestration loop** — predicate-bearing archetypes (ship-ready, optimize-metric, fix-broken, harden, build-feature, explore). Goal has a mechanical Success predicate; the loop runs until that predicate is met. - **Single-pass dispatch** — subjective/terminal archetypes (document, what-to-build, decide-design). Routes once to the fitting subcommand (learn / improve / reason), lets it self-terminate, then reports. No loop, no Plateau, no ship gate. diff --git a/.claude/skills/autoresearch/references/behavior-preserving-optimization.md b/.claude/skills/autoresearch/references/behavior-preserving-optimization.md new file mode 100644 index 0000000..d3271cf --- /dev/null +++ b/.claude/skills/autoresearch/references/behavior-preserving-optimization.md @@ -0,0 +1,61 @@ +# Behavior-Preserving Performance Optimization + +Load this reference when the objective is performance, latency, throughput, +runtime, CPU, memory, allocations, binary size, power, energy, bandwidth, or +compute cost and the system must keep doing the same useful job. + +## Separate the Search Signal from Acceptance + +The metric being optimized cannot also prove behavior preservation. A candidate +can become faster by skipping work, narrowing inputs, lowering quality, moving +cost elsewhere, or changing timing outside the benchmark's view. + +Before the baseline, define: + +- the performance objective, workload, measurement method, and smallest useful + gain; +- user-visible, protocol, perceptual, or numerical behavior that must remain; +- independent behavior checks that reject at least one known-bad change; +- resource boundaries for memory, startup, energy, bandwidth, deadline misses, + and tail latency where relevant; +- comparable artifact, configuration, hardware, and input provenance; +- representative inputs for search and separate held-out or adversarial inputs + for confirmation; +- a rollback point. + +If a guard was chosen mainly because it is easy to keep green, replace it. + +## Run Two Loops + +Use the normal autoresearch loop for exploration. Reversible batches may locate +promising directions, and the working metric may rank them. A `keep` decision +means only "better exploratory incumbent"; it does not establish causality or +readiness. + +Confirm each promising direction before calling it behavior-preserving: + +1. Rebuild it as an isolated, reviewable change. +2. Re-run the objective and independent behavior checks on representative plus + held-out or adversarial workloads. +3. Compare end-to-end and tail costs, not only the edited function or average. +4. Reject the candidate when any behavior or resource boundary fails. +5. Preserve raw baseline/candidate evidence and comparable provenance. +6. Record unmeasured behavior and the rollback path. + +## Resist Proxy Success + +A green metric or guard proves only that the declared observations passed. It +does not prove that the observations represent the user's real objective, that +the measurements are truthful, or that no important behavior is missing. + +Before final acceptance, a reviewer should be able to identify the causal +change, inspect the raw evidence, see an independent check reject a known-bad +case, and explain what remains unmeasured. + +Reject these arguments: + +- "The benchmark is much faster, so the change is good." +- "The outputs look close enough." +- "The guard passed, so behavior is preserved." +- "We can isolate the five tweaks after they land." +- "The benchmark input is representative by definition." diff --git a/.opencode/commands/autoresearch.md b/.opencode/commands/autoresearch.md index 3588289..e2f9317 100644 --- a/.opencode/commands/autoresearch.md +++ b/.opencode/commands/autoresearch.md @@ -29,6 +29,15 @@ If Goal, Scope, Metric, or Verify missing → use question (single batched call) Q4 (Guard): "Safety command that must always pass?" — options: test cmd, build cmd, skip If ALL provided inline → skip setup, proceed directly. +## Conditional Performance Reference + +If Goal or Metric targets performance, latency, throughput, runtime, CPU, +memory, allocations, binary size, power, energy, bandwidth, or compute cost, read +`.opencode/skills/autoresearch/references/behavior-preserving-optimization.md` +before establishing the baseline. Apply its exploration/confirmation split: a +metric-improving `keep` is an exploratory incumbent, not proof that behavior is +preserved. + ## Precondition Checks 1. Verify git repo exists (`git rev-parse --git-dir`) diff --git a/.opencode/commands/autoresearch_plan.md b/.opencode/commands/autoresearch_plan.md index 8f80ce3..12b960e 100644 --- a/.opencode/commands/autoresearch_plan.md +++ b/.opencode/commands/autoresearch_plan.md @@ -43,6 +43,12 @@ For metric-driven goals: - Determine direction: higher_is_better or lower_is_better - Propose metric name and description +For performance, latency, throughput, runtime, CPU, memory, allocations, binary +size, power, energy, bandwidth, or compute-cost goals, read +`.opencode/skills/autoresearch/references/behavior-preserving-optimization.md`. +Derive a behavior/resource Guard and a held-out confirmation plan separately +from the headline metric; do not let the optimized metric certify itself. + For subjective goals: - Suggest proxy metrics where possible - Or recommend /autoresearch_reason for non-measurable goals diff --git a/.opencode/skills/autoresearch/SKILL.md b/.opencode/skills/autoresearch/SKILL.md index b8b1e32..6ff6f09 100644 --- a/.opencode/skills/autoresearch/SKILL.md +++ b/.opencode/skills/autoresearch/SKILL.md @@ -64,6 +64,11 @@ Print a banner on every invocation: `[autoresearch] mode: classic | orchestrator Activated when a plain-language goal is given without `Metric:`/`Verify:`. Classifies the goal into a **Goal archetype** — see `references/orchestrator-routing.md` for the archetype table and router decision table. +For performance, latency, throughput, runtime, CPU, memory, allocations, binary +size, power, energy, bandwidth, or compute-cost goals, read +`references/behavior-preserving-optimization.md`. The working metric ranks +candidates; independent behavior evidence decides whether a candidate survives. + **Two modes based on archetype:** - **Orchestration loop** — predicate-bearing archetypes (ship-ready, optimize-metric, fix-broken, harden, build-feature, explore). Goal has a mechanical Success predicate; the loop runs until that predicate is met. - **Single-pass dispatch** — subjective/terminal archetypes (document, what-to-build, decide-design). Routes once to the fitting subcommand (learn / improve / reason), lets it self-terminate, then reports. No loop, no Plateau, no ship gate. diff --git a/.opencode/skills/autoresearch/references/behavior-preserving-optimization.md b/.opencode/skills/autoresearch/references/behavior-preserving-optimization.md new file mode 100644 index 0000000..d3271cf --- /dev/null +++ b/.opencode/skills/autoresearch/references/behavior-preserving-optimization.md @@ -0,0 +1,61 @@ +# Behavior-Preserving Performance Optimization + +Load this reference when the objective is performance, latency, throughput, +runtime, CPU, memory, allocations, binary size, power, energy, bandwidth, or +compute cost and the system must keep doing the same useful job. + +## Separate the Search Signal from Acceptance + +The metric being optimized cannot also prove behavior preservation. A candidate +can become faster by skipping work, narrowing inputs, lowering quality, moving +cost elsewhere, or changing timing outside the benchmark's view. + +Before the baseline, define: + +- the performance objective, workload, measurement method, and smallest useful + gain; +- user-visible, protocol, perceptual, or numerical behavior that must remain; +- independent behavior checks that reject at least one known-bad change; +- resource boundaries for memory, startup, energy, bandwidth, deadline misses, + and tail latency where relevant; +- comparable artifact, configuration, hardware, and input provenance; +- representative inputs for search and separate held-out or adversarial inputs + for confirmation; +- a rollback point. + +If a guard was chosen mainly because it is easy to keep green, replace it. + +## Run Two Loops + +Use the normal autoresearch loop for exploration. Reversible batches may locate +promising directions, and the working metric may rank them. A `keep` decision +means only "better exploratory incumbent"; it does not establish causality or +readiness. + +Confirm each promising direction before calling it behavior-preserving: + +1. Rebuild it as an isolated, reviewable change. +2. Re-run the objective and independent behavior checks on representative plus + held-out or adversarial workloads. +3. Compare end-to-end and tail costs, not only the edited function or average. +4. Reject the candidate when any behavior or resource boundary fails. +5. Preserve raw baseline/candidate evidence and comparable provenance. +6. Record unmeasured behavior and the rollback path. + +## Resist Proxy Success + +A green metric or guard proves only that the declared observations passed. It +does not prove that the observations represent the user's real objective, that +the measurements are truthful, or that no important behavior is missing. + +Before final acceptance, a reviewer should be able to identify the causal +change, inspect the raw evidence, see an independent check reject a known-bad +case, and explain what remains unmeasured. + +Reject these arguments: + +- "The benchmark is much faster, so the change is good." +- "The outputs look close enough." +- "The guard passed, so behavior is preserved." +- "We can isolate the five tweaks after they land." +- "The benchmark input is representative by definition." diff --git a/AGENTS.md b/AGENTS.md index 2bb2f91..fbfcda6 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -261,7 +261,7 @@ autoresearch:probe --chain reason # interrogate → debate → conve 1. **Loop until done** — unbounded: forever. Bounded: N times then summarize. 2. **Read before write** — understand full context before modifying. 3. **One change per iteration** — atomic changes. If it breaks, you know why. -4. **Mechanical verification only** — no subjective "looks good." Use metrics. +4. **Mechanical search signal, independent acceptance** — no subjective "looks good," but do not let the optimized metric certify behavior preservation. 5. **Automatic rollback** — failed changes revert instantly via `git revert`. 6. **Simplicity wins** — equal results + less code = KEEP. 7. **Git is memory** — experiments committed with `experiment:` prefix, agent reads `git log` + `git diff` before each iteration. diff --git a/plugins/autoresearch/skills/autoresearch/SKILL.md b/plugins/autoresearch/skills/autoresearch/SKILL.md index d8e4841..eeec57b 100644 --- a/plugins/autoresearch/skills/autoresearch/SKILL.md +++ b/plugins/autoresearch/skills/autoresearch/SKILL.md @@ -64,6 +64,11 @@ Print a banner on every invocation: `[autoresearch] mode: classic | orchestrator Activated when a plain-language goal is given without `Metric:`/`Verify:`. Classifies the goal into a **Goal archetype** — see `references/orchestrator-routing.md` for the archetype table and router decision table. +For performance, latency, throughput, runtime, CPU, memory, allocations, binary +size, power, energy, bandwidth, or compute-cost goals, read +`references/behavior-preserving-optimization.md`. The working metric ranks +candidates; independent behavior evidence decides whether a candidate survives. + **Two modes based on archetype:** - **Orchestration loop** — predicate-bearing archetypes (ship-ready, optimize-metric, fix-broken, harden, build-feature, explore). Goal has a mechanical Success predicate; the loop runs until that predicate is met. - **Single-pass dispatch** — subjective/terminal archetypes (document, what-to-build, decide-design). Routes once to the fitting subcommand (learn / improve / reason), lets it self-terminate, then reports. No loop, no Plateau, no ship gate. diff --git a/plugins/autoresearch/skills/autoresearch/autoresearch.md b/plugins/autoresearch/skills/autoresearch/autoresearch.md index 7eb1cf1..aad1df8 100644 --- a/plugins/autoresearch/skills/autoresearch/autoresearch.md +++ b/plugins/autoresearch/skills/autoresearch/autoresearch.md @@ -29,6 +29,15 @@ If Goal, Scope, Metric, or Verify missing → use request_user_input (single bat Q4 (Guard): "Safety command that must always pass?" — options: test cmd, build cmd, skip If ALL provided inline → skip setup, proceed directly. +## Conditional Performance Reference + +If Goal or Metric targets performance, latency, throughput, runtime, CPU, +memory, allocations, binary size, power, energy, bandwidth, or compute cost, read +`.claude/skills$autoresearch/references/behavior-preserving-optimization.md` +before establishing the baseline. Apply its exploration/confirmation split: a +metric-improving `keep` is an exploratory incumbent, not proof that behavior is +preserved. + ## Precondition Checks 1. Verify git repo exists (`git rev-parse --git-dir`) diff --git a/plugins/autoresearch/skills/autoresearch/plan.md b/plugins/autoresearch/skills/autoresearch/plan.md index eed5438..de785da 100644 --- a/plugins/autoresearch/skills/autoresearch/plan.md +++ b/plugins/autoresearch/skills/autoresearch/plan.md @@ -43,6 +43,12 @@ For metric-driven goals: - Determine direction: higher_is_better or lower_is_better - Propose metric name and description +For performance, latency, throughput, runtime, CPU, memory, allocations, binary +size, power, energy, bandwidth, or compute-cost goals, read +`.claude/skills$autoresearch/references/behavior-preserving-optimization.md`. +Derive a behavior/resource Guard and a held-out confirmation plan separately +from the headline metric; do not let the optimized metric certify itself. + For subjective goals: - Suggest proxy metrics where possible - Or recommend $autoresearch reason for non-measurable goals diff --git a/plugins/autoresearch/skills/autoresearch/references/behavior-preserving-optimization.md b/plugins/autoresearch/skills/autoresearch/references/behavior-preserving-optimization.md new file mode 100644 index 0000000..d3271cf --- /dev/null +++ b/plugins/autoresearch/skills/autoresearch/references/behavior-preserving-optimization.md @@ -0,0 +1,61 @@ +# Behavior-Preserving Performance Optimization + +Load this reference when the objective is performance, latency, throughput, +runtime, CPU, memory, allocations, binary size, power, energy, bandwidth, or +compute cost and the system must keep doing the same useful job. + +## Separate the Search Signal from Acceptance + +The metric being optimized cannot also prove behavior preservation. A candidate +can become faster by skipping work, narrowing inputs, lowering quality, moving +cost elsewhere, or changing timing outside the benchmark's view. + +Before the baseline, define: + +- the performance objective, workload, measurement method, and smallest useful + gain; +- user-visible, protocol, perceptual, or numerical behavior that must remain; +- independent behavior checks that reject at least one known-bad change; +- resource boundaries for memory, startup, energy, bandwidth, deadline misses, + and tail latency where relevant; +- comparable artifact, configuration, hardware, and input provenance; +- representative inputs for search and separate held-out or adversarial inputs + for confirmation; +- a rollback point. + +If a guard was chosen mainly because it is easy to keep green, replace it. + +## Run Two Loops + +Use the normal autoresearch loop for exploration. Reversible batches may locate +promising directions, and the working metric may rank them. A `keep` decision +means only "better exploratory incumbent"; it does not establish causality or +readiness. + +Confirm each promising direction before calling it behavior-preserving: + +1. Rebuild it as an isolated, reviewable change. +2. Re-run the objective and independent behavior checks on representative plus + held-out or adversarial workloads. +3. Compare end-to-end and tail costs, not only the edited function or average. +4. Reject the candidate when any behavior or resource boundary fails. +5. Preserve raw baseline/candidate evidence and comparable provenance. +6. Record unmeasured behavior and the rollback path. + +## Resist Proxy Success + +A green metric or guard proves only that the declared observations passed. It +does not prove that the observations represent the user's real objective, that +the measurements are truthful, or that no important behavior is missing. + +Before final acceptance, a reviewer should be able to identify the causal +change, inspect the raw evidence, see an independent check reject a known-bad +case, and explain what remains unmeasured. + +Reject these arguments: + +- "The benchmark is much faster, so the change is good." +- "The outputs look close enough." +- "The guard passed, so behavior is preserved." +- "We can isolate the five tweaks after they land." +- "The benchmark input is representative by definition."