Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .agents/skills/autoresearch/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,11 @@ Print a banner on every invocation: `[autoresearch] mode: classic | orchestrator

Activated when a plain-language goal is given without `Metric:`/`Verify:`. Classifies the goal into a **Goal archetype** — see `references/orchestrator-routing.md` for the archetype table and router decision table.

For performance, latency, throughput, runtime, CPU, memory, allocations, binary
size, power, energy, bandwidth, or compute-cost goals, read
`references/behavior-preserving-optimization.md`. The working metric ranks
candidates; independent behavior evidence decides whether a candidate survives.

**Two modes based on archetype:**
- **Orchestration loop** — predicate-bearing archetypes (ship-ready, optimize-metric, fix-broken, harden, build-feature, explore). Goal has a mechanical Success predicate; the loop runs until that predicate is met.
- **Single-pass dispatch** — subjective/terminal archetypes (document, what-to-build, decide-design). Routes once to the fitting subcommand (learn / improve / reason), lets it self-terminate, then reports. No loop, no Plateau, no ship gate.
Expand Down
9 changes: 9 additions & 0 deletions .agents/skills/autoresearch/autoresearch.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,15 @@ If Goal, Scope, Metric, or Verify missing → use request_user_input (single bat
Q4 (Guard): "Safety command that must always pass?" — options: test cmd, build cmd, skip
If ALL provided inline → skip setup, proceed directly.

## Conditional Performance Reference

If Goal or Metric targets performance, latency, throughput, runtime, CPU,
memory, allocations, binary size, power, energy, bandwidth, or compute cost, read
`.claude/skills$autoresearch/references/behavior-preserving-optimization.md`
before establishing the baseline. Apply its exploration/confirmation split: a
metric-improving `keep` is an exploratory incumbent, not proof that behavior is
preserved.

## Precondition Checks

1. Verify git repo exists (`git rev-parse --git-dir`)
Expand Down
6 changes: 6 additions & 0 deletions .agents/skills/autoresearch/plan.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,12 @@ For metric-driven goals:
- Determine direction: higher_is_better or lower_is_better
- Propose metric name and description

For performance, latency, throughput, runtime, CPU, memory, allocations, binary
size, power, energy, bandwidth, or compute-cost goals, read
`.claude/skills$autoresearch/references/behavior-preserving-optimization.md`.
Derive a behavior/resource Guard and a held-out confirmation plan separately
from the headline metric; do not let the optimized metric certify itself.

For subjective goals:
- Suggest proxy metrics where possible
- Or recommend $autoresearch reason for non-measurable goals
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
# Behavior-Preserving Performance Optimization

Load this reference when the objective is performance, latency, throughput,
runtime, CPU, memory, allocations, binary size, power, energy, bandwidth, or
compute cost and the system must keep doing the same useful job.

## Separate the Search Signal from Acceptance

The metric being optimized cannot also prove behavior preservation. A candidate
can become faster by skipping work, narrowing inputs, lowering quality, moving
cost elsewhere, or changing timing outside the benchmark's view.

Before the baseline, define:

- the performance objective, workload, measurement method, and smallest useful
gain;
- user-visible, protocol, perceptual, or numerical behavior that must remain;
- independent behavior checks that reject at least one known-bad change;
- resource boundaries for memory, startup, energy, bandwidth, deadline misses,
and tail latency where relevant;
- comparable artifact, configuration, hardware, and input provenance;
- representative inputs for search and separate held-out or adversarial inputs
for confirmation;
- a rollback point.

If a guard was chosen mainly because it is easy to keep green, replace it.

## Run Two Loops

Use the normal autoresearch loop for exploration. Reversible batches may locate
promising directions, and the working metric may rank them. A `keep` decision
means only "better exploratory incumbent"; it does not establish causality or
readiness.

Confirm each promising direction before calling it behavior-preserving:

1. Rebuild it as an isolated, reviewable change.
2. Re-run the objective and independent behavior checks on representative plus
held-out or adversarial workloads.
3. Compare end-to-end and tail costs, not only the edited function or average.
4. Reject the candidate when any behavior or resource boundary fails.
5. Preserve raw baseline/candidate evidence and comparable provenance.
6. Record unmeasured behavior and the rollback path.

## Resist Proxy Success

A green metric or guard proves only that the declared observations passed. It
does not prove that the observations represent the user's real objective, that
the measurements are truthful, or that no important behavior is missing.

Before final acceptance, a reviewer should be able to identify the causal
change, inspect the raw evidence, see an independent check reject a known-bad
case, and explain what remains unmeasured.

Reject these arguments:

- "The benchmark is much faster, so the change is good."
- "The outputs look close enough."
- "The guard passed, so behavior is preserved."
- "We can isolate the five tweaks after they land."
- "The benchmark input is representative by definition."
9 changes: 9 additions & 0 deletions .claude/commands/autoresearch.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,15 @@ If Goal, Scope, Metric, or Verify missing → use AskUserQuestion (single batche
Q4 (Guard): "Safety command that must always pass?" — options: test cmd, build cmd, skip
If ALL provided inline → skip setup, proceed directly.

## Conditional Performance Reference

If Goal or Metric targets performance, latency, throughput, runtime, CPU,
memory, allocations, binary size, power, energy, bandwidth, or compute cost, read
`.claude/skills/autoresearch/references/behavior-preserving-optimization.md`
before establishing the baseline. Apply its exploration/confirmation split: a
metric-improving `keep` is an exploratory incumbent, not proof that behavior is
preserved.

## Precondition Checks

1. Verify git repo exists (`git rev-parse --git-dir`)
Expand Down
6 changes: 6 additions & 0 deletions .claude/commands/autoresearch/plan.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,12 @@ For metric-driven goals:
- Determine direction: higher_is_better or lower_is_better
- Propose metric name and description

For performance, latency, throughput, runtime, CPU, memory, allocations, binary
size, power, energy, bandwidth, or compute-cost goals, read
`.claude/skills/autoresearch/references/behavior-preserving-optimization.md`.
Derive a behavior/resource Guard and a held-out confirmation plan separately
from the headline metric; do not let the optimized metric certify itself.

For subjective goals:
- Suggest proxy metrics where possible
- Or recommend /autoresearch:reason for non-measurable goals
Expand Down
5 changes: 5 additions & 0 deletions .claude/skills/autoresearch/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,11 @@ Print a banner on every invocation: `[autoresearch] mode: classic | orchestrator

Activated when a plain-language goal is given without `Metric:`/`Verify:`. Classifies the goal into a **Goal archetype** — see `references/orchestrator-routing.md` for the archetype table and router decision table.

For performance, latency, throughput, runtime, CPU, memory, allocations, binary
size, power, energy, bandwidth, or compute-cost goals, read
`references/behavior-preserving-optimization.md`. The working metric ranks
candidates; independent behavior evidence decides whether a candidate survives.

**Two modes based on archetype:**
- **Orchestration loop** — predicate-bearing archetypes (ship-ready, optimize-metric, fix-broken, harden, build-feature, explore). Goal has a mechanical Success predicate; the loop runs until that predicate is met.
- **Single-pass dispatch** — subjective/terminal archetypes (document, what-to-build, decide-design). Routes once to the fitting subcommand (learn / improve / reason), lets it self-terminate, then reports. No loop, no Plateau, no ship gate.
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
# Behavior-Preserving Performance Optimization

Load this reference when the objective is performance, latency, throughput,
runtime, CPU, memory, allocations, binary size, power, energy, bandwidth, or
compute cost and the system must keep doing the same useful job.

## Separate the Search Signal from Acceptance

The metric being optimized cannot also prove behavior preservation. A candidate
can become faster by skipping work, narrowing inputs, lowering quality, moving
cost elsewhere, or changing timing outside the benchmark's view.

Before the baseline, define:

- the performance objective, workload, measurement method, and smallest useful
gain;
- user-visible, protocol, perceptual, or numerical behavior that must remain;
- independent behavior checks that reject at least one known-bad change;
- resource boundaries for memory, startup, energy, bandwidth, deadline misses,
and tail latency where relevant;
- comparable artifact, configuration, hardware, and input provenance;
- representative inputs for search and separate held-out or adversarial inputs
for confirmation;
- a rollback point.

If a guard was chosen mainly because it is easy to keep green, replace it.

## Run Two Loops

Use the normal autoresearch loop for exploration. Reversible batches may locate
promising directions, and the working metric may rank them. A `keep` decision
means only "better exploratory incumbent"; it does not establish causality or
readiness.

Confirm each promising direction before calling it behavior-preserving:

1. Rebuild it as an isolated, reviewable change.
2. Re-run the objective and independent behavior checks on representative plus
held-out or adversarial workloads.
3. Compare end-to-end and tail costs, not only the edited function or average.
4. Reject the candidate when any behavior or resource boundary fails.
5. Preserve raw baseline/candidate evidence and comparable provenance.
6. Record unmeasured behavior and the rollback path.

## Resist Proxy Success

A green metric or guard proves only that the declared observations passed. It
does not prove that the observations represent the user's real objective, that
the measurements are truthful, or that no important behavior is missing.

Before final acceptance, a reviewer should be able to identify the causal
change, inspect the raw evidence, see an independent check reject a known-bad
case, and explain what remains unmeasured.

Reject these arguments:

- "The benchmark is much faster, so the change is good."
- "The outputs look close enough."
- "The guard passed, so behavior is preserved."
- "We can isolate the five tweaks after they land."
- "The benchmark input is representative by definition."
9 changes: 9 additions & 0 deletions .opencode/commands/autoresearch.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,15 @@ If Goal, Scope, Metric, or Verify missing → use question (single batched call)
Q4 (Guard): "Safety command that must always pass?" — options: test cmd, build cmd, skip
If ALL provided inline → skip setup, proceed directly.

## Conditional Performance Reference

If Goal or Metric targets performance, latency, throughput, runtime, CPU,
memory, allocations, binary size, power, energy, bandwidth, or compute cost, read
`.opencode/skills/autoresearch/references/behavior-preserving-optimization.md`
before establishing the baseline. Apply its exploration/confirmation split: a
metric-improving `keep` is an exploratory incumbent, not proof that behavior is
preserved.

## Precondition Checks

1. Verify git repo exists (`git rev-parse --git-dir`)
Expand Down
6 changes: 6 additions & 0 deletions .opencode/commands/autoresearch_plan.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,12 @@ For metric-driven goals:
- Determine direction: higher_is_better or lower_is_better
- Propose metric name and description

For performance, latency, throughput, runtime, CPU, memory, allocations, binary
size, power, energy, bandwidth, or compute-cost goals, read
`.opencode/skills/autoresearch/references/behavior-preserving-optimization.md`.
Derive a behavior/resource Guard and a held-out confirmation plan separately
from the headline metric; do not let the optimized metric certify itself.

For subjective goals:
- Suggest proxy metrics where possible
- Or recommend /autoresearch_reason for non-measurable goals
Expand Down
5 changes: 5 additions & 0 deletions .opencode/skills/autoresearch/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,11 @@ Print a banner on every invocation: `[autoresearch] mode: classic | orchestrator

Activated when a plain-language goal is given without `Metric:`/`Verify:`. Classifies the goal into a **Goal archetype** — see `references/orchestrator-routing.md` for the archetype table and router decision table.

For performance, latency, throughput, runtime, CPU, memory, allocations, binary
size, power, energy, bandwidth, or compute-cost goals, read
`references/behavior-preserving-optimization.md`. The working metric ranks
candidates; independent behavior evidence decides whether a candidate survives.

**Two modes based on archetype:**
- **Orchestration loop** — predicate-bearing archetypes (ship-ready, optimize-metric, fix-broken, harden, build-feature, explore). Goal has a mechanical Success predicate; the loop runs until that predicate is met.
- **Single-pass dispatch** — subjective/terminal archetypes (document, what-to-build, decide-design). Routes once to the fitting subcommand (learn / improve / reason), lets it self-terminate, then reports. No loop, no Plateau, no ship gate.
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
# Behavior-Preserving Performance Optimization

Load this reference when the objective is performance, latency, throughput,
runtime, CPU, memory, allocations, binary size, power, energy, bandwidth, or
compute cost and the system must keep doing the same useful job.

## Separate the Search Signal from Acceptance

The metric being optimized cannot also prove behavior preservation. A candidate
can become faster by skipping work, narrowing inputs, lowering quality, moving
cost elsewhere, or changing timing outside the benchmark's view.

Before the baseline, define:

- the performance objective, workload, measurement method, and smallest useful
gain;
- user-visible, protocol, perceptual, or numerical behavior that must remain;
- independent behavior checks that reject at least one known-bad change;
- resource boundaries for memory, startup, energy, bandwidth, deadline misses,
and tail latency where relevant;
- comparable artifact, configuration, hardware, and input provenance;
- representative inputs for search and separate held-out or adversarial inputs
for confirmation;
- a rollback point.

If a guard was chosen mainly because it is easy to keep green, replace it.

## Run Two Loops

Use the normal autoresearch loop for exploration. Reversible batches may locate
promising directions, and the working metric may rank them. A `keep` decision
means only "better exploratory incumbent"; it does not establish causality or
readiness.

Confirm each promising direction before calling it behavior-preserving:

1. Rebuild it as an isolated, reviewable change.
2. Re-run the objective and independent behavior checks on representative plus
held-out or adversarial workloads.
3. Compare end-to-end and tail costs, not only the edited function or average.
4. Reject the candidate when any behavior or resource boundary fails.
5. Preserve raw baseline/candidate evidence and comparable provenance.
6. Record unmeasured behavior and the rollback path.

## Resist Proxy Success

A green metric or guard proves only that the declared observations passed. It
does not prove that the observations represent the user's real objective, that
the measurements are truthful, or that no important behavior is missing.

Before final acceptance, a reviewer should be able to identify the causal
change, inspect the raw evidence, see an independent check reject a known-bad
case, and explain what remains unmeasured.

Reject these arguments:

- "The benchmark is much faster, so the change is good."
- "The outputs look close enough."
- "The guard passed, so behavior is preserved."
- "We can isolate the five tweaks after they land."
- "The benchmark input is representative by definition."
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -261,7 +261,7 @@ autoresearch:probe --chain reason # interrogate → debate → conve
1. **Loop until done** — unbounded: forever. Bounded: N times then summarize.
2. **Read before write** — understand full context before modifying.
3. **One change per iteration** — atomic changes. If it breaks, you know why.
4. **Mechanical verification only** — no subjective "looks good." Use metrics.
4. **Mechanical search signal, independent acceptance** — no subjective "looks good," but do not let the optimized metric certify behavior preservation.
5. **Automatic rollback** — failed changes revert instantly via `git revert`.
6. **Simplicity wins** — equal results + less code = KEEP.
7. **Git is memory** — experiments committed with `experiment:` prefix, agent reads `git log` + `git diff` before each iteration.
Expand Down
5 changes: 5 additions & 0 deletions plugins/autoresearch/skills/autoresearch/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,11 @@ Print a banner on every invocation: `[autoresearch] mode: classic | orchestrator

Activated when a plain-language goal is given without `Metric:`/`Verify:`. Classifies the goal into a **Goal archetype** — see `references/orchestrator-routing.md` for the archetype table and router decision table.

For performance, latency, throughput, runtime, CPU, memory, allocations, binary
size, power, energy, bandwidth, or compute-cost goals, read
`references/behavior-preserving-optimization.md`. The working metric ranks
candidates; independent behavior evidence decides whether a candidate survives.

**Two modes based on archetype:**
- **Orchestration loop** — predicate-bearing archetypes (ship-ready, optimize-metric, fix-broken, harden, build-feature, explore). Goal has a mechanical Success predicate; the loop runs until that predicate is met.
- **Single-pass dispatch** — subjective/terminal archetypes (document, what-to-build, decide-design). Routes once to the fitting subcommand (learn / improve / reason), lets it self-terminate, then reports. No loop, no Plateau, no ship gate.
Expand Down
9 changes: 9 additions & 0 deletions plugins/autoresearch/skills/autoresearch/autoresearch.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,15 @@ If Goal, Scope, Metric, or Verify missing → use request_user_input (single bat
Q4 (Guard): "Safety command that must always pass?" — options: test cmd, build cmd, skip
If ALL provided inline → skip setup, proceed directly.

## Conditional Performance Reference

If Goal or Metric targets performance, latency, throughput, runtime, CPU,
memory, allocations, binary size, power, energy, bandwidth, or compute cost, read
`.claude/skills$autoresearch/references/behavior-preserving-optimization.md`
before establishing the baseline. Apply its exploration/confirmation split: a
metric-improving `keep` is an exploratory incumbent, not proof that behavior is
preserved.

## Precondition Checks

1. Verify git repo exists (`git rev-parse --git-dir`)
Expand Down
6 changes: 6 additions & 0 deletions plugins/autoresearch/skills/autoresearch/plan.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,12 @@ For metric-driven goals:
- Determine direction: higher_is_better or lower_is_better
- Propose metric name and description

For performance, latency, throughput, runtime, CPU, memory, allocations, binary
size, power, energy, bandwidth, or compute-cost goals, read
`.claude/skills$autoresearch/references/behavior-preserving-optimization.md`.
Derive a behavior/resource Guard and a held-out confirmation plan separately
from the headline metric; do not let the optimized metric certify itself.

For subjective goals:
- Suggest proxy metrics where possible
- Or recommend $autoresearch reason for non-measurable goals
Expand Down
Loading