Add rfe-creator as a dispatched benchmark

[OsherElhadad]

Summary

Wires the RFE-Creator skill pipeline into the same run_suite.sh / benchmarks.yml path as tau2/swebench/skillsbench/spreadsheetbench, rather than leaving it as a one-off local run:

• templates/adapters/rfe_creator/adapter.py — the CapabilityAdapter (tasks, run_target, score, live), ported from the original prototype and parameterized by env vars instead of hardcoded laptop paths. Drops _force_subagent_model(), which existed only to fight a global ~/.claude/settings.json that CI does not have.
• ci/benchmarks/rfe-creator/{smoke,full}/tasks.json — 5 headroom-calibrated cases / all 25 curated cases (ids only), selected from that run's own per-task baseline rewards (lowest-reward = most headroom).
• reward_overlay.yaml + utils/fetch_data.sh — both upstream repos (opendatahub-io/rfe-creator, opendatahub-io/agent-eval-harness) are public but unlicensed, so neither their code nor the 25 eval cases are vendored. fetch_data.sh clones both at run time and merges this repo's own small reward_overlay.yaml onto the upstream eval.yaml — same convention as spreadsheetbench/fetch_data.sh and the parsec local-shadow pattern.
• run_suite.sh / ci_setup.sh — new rfe-creator case arm, using the same ANTHROPIC_BASE_URL/ANTHROPIC_AUTH_TOKEN gateway every other bench already uses (the self-hosted ibm-vpc runner exists precisely because that gateway is VPC-internal — not a new requirement for this bench).
• benchmarks.yml / site/benchmarks.js — added to BENCHES, the workflow_dispatch options, and the live-panel JOB_RE.
• core/tests/test_benchmarks_plan_legs.py, test_site_live_panel_tiers.py — added to the two rosters those tests assert against.
• Docs: ci/benchmarks/README.md (suite list, runner prerequisites, cost caveat), new ci/benchmarks/rfe-creator/README.md (licensing rationale, cost, pointer to the original run), templates/adapters/README.md, docs/ADAPTER_TEMPLATES.md, site/benchmarks.html, site/adapter-templates.html.

What this does not do

• No pilot/ tier — test_benchmarks_plan_legs.py asserts the pilot roster is exactly [spreadsheetbench, swebench]; adding one would break that test.
• No benchmark-history record or dashboard row. That requires an actual CI dispatch producing a sealed final.json — this PR only adds the wiring. The original run this bench's numbers come from (baseline 0.897 -> champion 0.925 val reward) is in-progress (3 of 10 iterations) and lives in the private cap-evolve-internal repo, not published as a benchmark result here. Follow-up: dispatch Actions -> Benchmarks -> rfe-creator / smoke (or full, given the cost below) once this merges.
• No vendored data. Both upstream repos are public but unlicensed — see ci/benchmarks/rfe-creator/README.md.

Cost note

A full run is not cheap: 25 tasks x trials x iterations measured $234.64 runner + $34.77 optimizer, 6.4 runner-hours for just 3 of 10 budgeted iterations on the original run. smoke (5 tasks) is the cheap regression signal.

Test plan

• python3 -m pytest core/tests/ -q — 1289 passed, 0 failed (no regressions from the roster/JOB_RE changes).
• bash -n on both modified shell scripts.
• utils/fetch_data.sh actually run end-to-end: clones both upstream repos, merges reward_overlay.yaml onto the fetched eval.yaml, produces a valid merged config.
• Both tasks.json files parse as JSON.
• Reviewer: dispatch Actions -> Benchmarks -> rfe-creator / smoke on the ibm-vpc runner to confirm the case arm actually runs end-to-end in CI (needs gateway access this sandbox doesn't have).

[skillberry-bot]

❌ Automatic Labeling Failed

An error occurred while trying to automatically label this pull request. Please check the workflow logs for details and add labels manually.