Add release train issue evaluation skill#4540
Conversation
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (3)
WalkthroughThis PR adds a new documentation-only agent skill, "react-on-rails-release-train-issue-evaluation," including its SKILL.md specification, an openai.yaml agent interface definition, and registration updates in AGENTS.md pointing to the new skill and shorthand invocation. ChangesRelease-Train Issue Evaluation Skill
Estimated code review effort: 1 (Trivial) | ~3 minutes Possibly related issues
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Review: Add release train issue evaluation skillOverview: Adds a new repo-local skill ( Verification performed
Code quality / style
Potential issues / risks
Test coverage
Security considerations
Verdict: Solid, low-risk, well-grounded addition. No blocking issues. |
|
@claude, review this PR |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
|
|
||
| 1. Refresh repo context: `git fetch --prune origin main`, then `.agents/bin/agent-workflow-seam-doctor`. Fetch the release branch if ancestry matters. | ||
| 2. Resolve candidates from live GitHub. For strict 48-hour windows, search by date then timestamp-filter locally because GitHub search is date-granular. | ||
| 3. Exclude started lanes. An issue is started when it has an assignee, linked/open implementation PR, private claim/heartbeat/branch, or implementation comment. Run `agent-coord doctor --json`, then targeted `agent-coord status --repo shakacode/react_on_rails --target <issue> --json`. Dead or expired claims count as started-but-stalled, not non-started. |
There was a problem hiding this comment.
Nit: this step invokes raw agent-coord doctor --json / agent-coord status ... --json, but every other repo-local workflow that touches agent-coord (.agents/workflows/pr-processing.md, .agents/workflows/post-merge-audit.md) routes these calls through the bounded wrapper (agent-coord-bounded, default 20s timeout via .agents/skills/pr-batch/bin/agent-coord-bounded) specifically to avoid an unbounded hang on a flaky/unavailable coordination backend. Worth aligning this skill with that convention, e.g.:
Run bounded `agent-coord doctor --json`, then bounded `agent-coord status --repo shakacode/react_on_rails --target <issue> --json`.
so an agent following this skill can't stall indefinitely if the private backend is unreachable.
Review summaryThis is a docs/process-only PR (already merged) that adds a new repo-local Codex/Claude skill for release-train issue triage and documents it in Findings:
Other observations (no action needed):
|
Greptile SummaryThis PR adds a new repo-local skill (
Confidence Score: 4/5Safe to merge — documentation and process files only, no runtime code touched. The change adds a new agent skill (Markdown + YAML) and updates AGENTS.md. The skill body is internally consistent and matches the conventions of existing repo-local skills. The only rough edges are the hardcoded release/17.0.0 in the SKILL.md frontmatter and openai.yaml default prompt (the body uses the parameterized form throughout), and a minor ambiguity around how agents should record started-but-stalled lanes in the output table. .agents/skills/react-on-rails-release-train-issue-evaluation/SKILL.md and agents/openai.yaml carry version-specific language that will drift as release lines change. Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[Issue candidate] --> B{Fetch repo context\ngit fetch + seam-doctor}
B --> C[Resolve candidates\nfrom live GitHub]
C --> D{Issue started?\nassignee / PR / claim / comment}
D -->|Yes - active| E[Exclude from evaluation]
D -->|Yes - stalled| F[started-but-stalled\nnot clearly excluded]
D -->|No| G[Evaluate with $evaluate-issue\nimpact · complexity · priority]
G --> H{Read release context\ntracker mode · labels · branch ancestry}
H --> I{Choose target}
I -->|RC regression / hard-gate\nblocker / security bug| J[release/X.Y.Z]
I -->|Depends on cherry-pick\nnot yet on release branch| K[release/X.Y.Z contingent]
I -->|Docs / tests / features\nnon-blocking hardening| L[main]
I -->|Speculative / duplicate\nnot worth doing| M[park / close]
J --> N[Output table\nwith UNKNOWN facts called out]
K --> N
L --> N
M --> N
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
A[Issue candidate] --> B{Fetch repo context\ngit fetch + seam-doctor}
B --> C[Resolve candidates\nfrom live GitHub]
C --> D{Issue started?\nassignee / PR / claim / comment}
D -->|Yes - active| E[Exclude from evaluation]
D -->|Yes - stalled| F[started-but-stalled\nnot clearly excluded]
D -->|No| G[Evaluate with $evaluate-issue\nimpact · complexity · priority]
G --> H{Read release context\ntracker mode · labels · branch ancestry}
H --> I{Choose target}
I -->|RC regression / hard-gate\nblocker / security bug| J[release/X.Y.Z]
I -->|Depends on cherry-pick\nnot yet on release branch| K[release/X.Y.Z contingent]
I -->|Docs / tests / features\nnon-blocking hardening| L[main]
I -->|Speculative / duplicate\nnot worth doing| M[park / close]
J --> N[Output table\nwith UNKNOWN facts called out]
K --> N
L --> N
M --> N
Reviews (1): Last reviewed commit: "Add release train issue evaluation skill" | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| name: react-on-rails-release-train-issue-evaluation | ||
| description: Use when evaluating React on Rails GitHub issues for release-train targeting, especially deciding whether new, follow-up, or non-started issues belong on release/17.0.0, another release/X.Y.Z branch, or main. | ||
| --- |
There was a problem hiding this comment.
description field names release/17.0.0 explicitly while the skill body consistently uses the parameterized form release/X.Y.Z throughout. When the team moves to 18.0.0 or another release line, the frontmatter will point agents at the wrong branch and could mislead skill-picker UI summaries that surface this description as context. A description referencing only release/X.Y.Z (dropping the specific version callout) would stay accurate across release trains without any follow-up edit.
| interface: | ||
| display_name: "React on Rails Release Train Issue Evaluation" | ||
| short_description: "Classify issues for release train or main" | ||
| default_prompt: "Use $react-on-rails-release-train-issue-evaluation to decide whether recent React on Rails issues belong on release/17.0.0 or main." |
There was a problem hiding this comment.
release/17.0.0 specificity appears here and will need a manual update every time the active release train advances. Using the form release/X.Y.Z (or deferring to the skill's own language) would make this durable. The other two fields (display_name, short_description) are already version-neutral.
|
|
||
| 1. Refresh repo context: `git fetch --prune origin main`, then `.agents/bin/agent-workflow-seam-doctor`. Fetch the release branch if ancestry matters. | ||
| 2. Resolve candidates from live GitHub. For strict 48-hour windows, search by date then timestamp-filter locally because GitHub search is date-granular. | ||
| 3. Exclude started lanes. An issue is started when it has an assignee, linked/open implementation PR, private claim/heartbeat/branch, or implementation comment. Run `agent-coord doctor --json`, then targeted `agent-coord status --repo shakacode/react_on_rails --target <issue> --json`. Dead or expired claims count as started-but-stalled, not non-started. |
There was a problem hiding this comment.
Started? column explain what an agent should record or recommend for stalled lanes. A stalled issue is not simply park/close (work was intended) but also isn't a clean candidate for a new target. Clarifying how to represent this state in the table — e.g., Yes (stalled) in the Started? column plus a Next action of "unblock or reassign" — would eliminate the ambiguity.
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
2 participants
Why
Recent React on Rails issue triage needs a reusable way to decide whether new or follow-up issues belong on the active 17.0.0 release train or should wait for main. The existing
evaluate-issueskill covers issue value, but not release-branch targeting.What changed
react-on-rails-release-train-issue-evaluationskill.AGENTS.mdalongside the existing issue evaluation workflow.Validation
git diff --check HEAD~1..HEADpython3 .../quick_validate.py .agents/skills/react-on-rails-release-train-issue-evaluation->Skill is valid!.agents/bin/validate --changed-> documentation-only, docs sidebar check passed, no further CI neededLabels: none; docs/process-only change.
Summary by CodeRabbit