v0.2 Roadmap: CodeRabbit-Class Reviewer Pilot

[100yenadmin]

Goal

Turn evaos-code-review-bot from the v0.1 safe ZCode inline reviewer into a CodeRabbit-class EVAOS reviewer with calibrated confidence, richer PR context, and low-noise repo policy controls.

Non-goals

• No auto-merge.
• No APPROVE reviews.
• No all-org rollout.
• No repo mutation by default review runs.
• No public 95% calibrated confidence until measured reliability supports it.

Current state

• v0.1 merged in PR Build ZCode PR review bot pilot #2 at merge commit 2dfad82600719c359e40df10f4ce0e92b5b7efbb.
• Launchd live canary is running from main with dryRun=false and canary PRs electricsheephq/WorldOS#1161, 100yenadmin/evaOS-GUI#497.
• Current bot deliberately disables native ZCode skills/MCP/memory and permits only read-only inspection tools.
• GitNexus is available locally for indexed repos but is not integrated into the bot.

Ordered work

1. Add CodeRabbit-style walkthrough and pre-merge summary.
2. Add repo policy config for review profile, path filters/instructions, labels/reviewers, checks, and finishing-touch commands.
3. Add read-only GitNexus context provider with stale/missing-index degraded mode.
4. Add curated read-only skill-pack prompt injection before any native skill: true enablement.
5. Add confidence calibration and public confidence display rules.
6. Add CodeRabbit/human comparison eval harness.
7. Add issue/PR enrichment lane.
8. Add opt-in finishing touches commands.

Acceptance criteria

• Each child issue has tests/evidence and no expansion of GitHub App permissions unless explicitly justified.
• The default review path remains read-only and App-authored.
• Duplicate suppression remains per {repo, pr, head_sha}.
• No secrets are written to comments, logs, or shareable evidence.
• P0/P1 REQUEST_CHANGES remains gated by validated findings.
• Public confidence labels say uncalibrated until calibration thresholds pass.

Validation / eval gates

• Eval required: yes
• Eval claim class: advisory
• Required eval suites: canary_shadow, historical_pr_replay, seeded_defect_recall, safety_redaction, duplicate_suppression, coderabbit_comparison
• Eval name/version: evaos-code-review-bot-v0.2-roadmap
• Dataset/scenario refs: WorldOS#1161, evaOS-GUI#497, historical CodeRabbit-reviewed PRs, seeded P0/P1 fixtures, negative-control docs-only PRs
• Baseline/comparison: CodeRabbit comments, human review outcomes, CI failures, merged-fix diffs
• Metrics and thresholds: 100% current-line validity; 100% secret redaction on seeded cases; zero mutation; no duplicate posting; P0/P1 calibrated precision Wilson lower bound >=95% before displaying 95% calibrated
• Runner/CI location: local focused eval packets under Lexar, with later GitHub Actions shadow mode if useful
• Failure owner: bot maintainer
• Eval evidence path: /Volumes/LEXAR/Codex/evals/zcode-glm-pr-review/<date>/<run-id>/
• Trace feedback target: this tracker, child issues, JSONL eval artifacts
• Eval proof boundary: proves pilot reviewer quality on scoped repos/datasets only; does not prove production all-org readiness or human-review replacement.

Evidence / resume links

• Merged v0.1 PR: Build ZCode PR review bot pilot #2
• v0.1 tracker: Pilot ZCode PR review bot for WorldOS and evaOS-GUI #1
• WorldOS canary review: feat(viewer): 4 sat→7 web-viewer UX/correctness fixes (battle-log sanitize, dice-header strip, roll context, drop-confirm) WorldOS#1161 (review)
• evaOS-GUI canary review: [codex] Guard public repo sensitive content 100yenadmin/evaOS-GUI#497 (review)
• Evidence root: /Volumes/LEXAR/Codex/evaos-code-review-bot/evidence/2026-07-01/

Current status

Created after v0.1 merge. Exact next action is to implement the walkthrough/pre-merge summary issue first while keeping live canary allowlist in place for 24h.

[100yenadmin]

Roadmap child issues created for milestone v0.2 CodeRabbit-Class Reviewer Pilot:

1. v0.2: CodeRabbit-style PR walkthrough and pre-merge summary #4 CodeRabbit-style PR walkthrough and pre-merge summary
2. v0.2: Repo policy config for profiles, paths, labels, reviewers, checks, and commands #5 Repo policy config for profiles, paths, labels, reviewers, checks, and commands
3. v0.2: Read-only GitNexus context provider #6 Read-only GitNexus context provider
4. v0.2: Curated read-only skill-pack prompt injection #7 Curated read-only skill-pack prompt injection
5. v0.2: Confidence calibration and public confidence display #8 Confidence calibration and public confidence display
6. v0.2: CodeRabbit and human review comparison eval harness #9 CodeRabbit and human review comparison eval harness
7. v0.2: Issue and PR enrichment lane #10 Issue and PR enrichment lane
8. v0.2: Opt-in finishing touches commands #11 Opt-in finishing touches commands

Execution note: keep the live canary allowlist in place for at least 24 hours after the PR #2 merge/sync. Do not expand App permissions or enable native ZCode skills/MCP/memory until the relevant child issue proves the safety gate.

[100yenadmin]

Current post-merge status:

• v0.1 PR Build ZCode PR review bot pilot #2 is merged at 2dfad82600719c359e40df10f4ce0e92b5b7efbb.
• Local checkout /Volumes/LEXAR/repos/evaos-code-review-bot is on main at the merge commit.
• launchd job com.electricsheephq.evaos-code-review-bot is running from the main checkout; post-restart pid observed as 92105.
• Post-merge live daemon cycles through cycle 5 completed with reviewed:0 and skippedProcessed:2.
• Milestone now has tracker v0.2 Roadmap: CodeRabbit-Class Reviewer Pilot #3 and child issues v0.2: CodeRabbit-style PR walkthrough and pre-merge summary #4-v0.2: Opt-in finishing touches commands #11.
• GitNexus refresh skipped: this repo is not currently registered in gitnexus list.

Exact next action: keep the canary allowlist in place for at least 24 hours, then start #4 (v0.2: CodeRabbit-style PR walkthrough and pre-merge summary) if the canary remains quiet. WorldOS#1161 and evaOS-GUI#497 remain separate follow-through PRs.

[100yenadmin]

24h canary heartbeat scheduled in Codex Desktop: evaos-code-review-bot-24h-canary-check, targeted for 2026-07-01T19:07:00Z. It will re-check launchd, logs, live DB rows, and duplicate bot reviews on the two canary PRs, then update this tracker with the result.