Skip to content

Add release train issue evaluation skill #4540

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
justin808 merged 1 commit into from
Jul 9, 2026
+54 −1
Merged

Add release train issue evaluation skill #4540

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 43 additions & 0 deletions .agents/skills/react-on-rails-release-train-issue-evaluation/SKILL.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
---
name: react-on-rails-release-train-issue-evaluation
description: Use when evaluating React on Rails GitHub issues for release-train targeting, especially deciding whether new, follow-up, or non-started issues belong on release/17.0.0, another release/X.Y.Z branch, or main.
---
Comment on lines +2 to +4

@greptile-apps greptile-apps Bot Jul 9, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Version hardcoded in frontmatter description — the description field names release/17.0.0 explicitly while the skill body consistently uses the parameterized form release/X.Y.Z throughout. When the team moves to 18.0.0 or another release line, the frontmatter will point agents at the wrong branch and could mislead skill-picker UI summaries that surface this description as context. A description referencing only release/X.Y.Z (dropping the specific version callout) would stay accurate across release trains without any follow-up edit.


# React on Rails Release Train Issue Evaluation

## Core Rule

Use this with `$evaluate-issue`. That skill answers "is this worth doing?"; this skill answers "does it belong on the active release train or on `main`?"

`release/*` receives only stabilizing fixes. Features, cleanup, docs, process work, non-blocking hardening, and ordinary follow-ups target `main`.

## Workflow

1. Refresh repo context: `git fetch --prune origin main`, then `.agents/bin/agent-workflow-seam-doctor`. Fetch the release branch if ancestry matters.
2. Resolve candidates from live GitHub. For strict 48-hour windows, search by date then timestamp-filter locally because GitHub search is date-granular.
3. Exclude started lanes. An issue is started when it has an assignee, linked/open implementation PR, private claim/heartbeat/branch, or implementation comment. Run `agent-coord doctor --json`, then targeted `agent-coord status --repo shakacode/react_on_rails --target <issue> --json`. Dead or expired claims count as started-but-stalled, not non-started.

@claude claude Bot Jul 9, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: this step invokes raw agent-coord doctor --json / agent-coord status ... --json, but every other repo-local workflow that touches agent-coord (.agents/workflows/pr-processing.md, .agents/workflows/post-merge-audit.md) routes these calls through the bounded wrapper (agent-coord-bounded, default 20s timeout via .agents/skills/pr-batch/bin/agent-coord-bounded) specifically to avoid an unbounded hang on a flaky/unavailable coordination backend. Worth aligning this skill with that convention, e.g.:

Run bounded `agent-coord doctor --json`, then bounded `agent-coord status --repo shakacode/react_on_rails --target <issue> --json`.

so an agent following this skill can't stall indefinitely if the private backend is unreachable.

@greptile-apps greptile-apps Bot Jul 9, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 "started-but-stalled" classification gap — step 3 introduces a distinct state ("Dead or expired claims count as started-but-stalled, not non-started") but neither step 6's disposition list nor the output table's Started? column explain what an agent should record or recommend for stalled lanes. A stalled issue is not simply park/close (work was intended) but also isn't a clean candidate for a new target. Clarifying how to represent this state in the table — e.g., Yes (stalled) in the Started? column plus a Next action of "unblock or reassign" — would eliminate the ambiguity.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

4. Read release context: active `Release gate:` tracker, `release` + `TRACKING` labels, the `Agent Release Mode` block, `agent-coord` phase when available, source PR base branches, and whether source commits are already on `origin/release/X.Y.Z`.
5. Evaluate each candidate with `$evaluate-issue`: evidence source, impact, complexity, process gap disposition, and priority.
6. Choose the target:
- `release/X.Y.Z`: verified RC/final stabilizer such as an RC regression, hard-gate failure, install/upgrade blocker, security/data-loss/wrong-output bug, release-branch CI/release tooling blocker, or final-only public API/breaking-change decision.
- `release/X.Y.Z contingent`: only needed if maintainers decide to cherry-pick a related main-only fix into the train. Name the dependency.
- `main`: docs, changelog for main-only PRs, CI/tooling hygiene, tests, process automation, non-blocking runtime hardening, features, and performance/cleanup without release-gate proof.
- `park/close`: P3, speculative, duplicate, no-PR evidence, or not worth doing.

Recommend labels/milestones/comments, but do not mutate GitHub unless the user explicitly authorizes writes.

## Output

Use a compact table:

| Issue | Started? | Evidence | Disposition | Target | Rationale | Next action |
| ----- | -------- | -------- | ----------- | ------ | --------- | ----------- |

Call out `UNKNOWN` facts explicitly, especially release tracker mode, source PR base, branch ancestry, and whether the issue is a confirmed RC regression.

## Common Mistakes

- Treating a title that says "before 17.0.0 final" as sufficient. Verify release-gate impact and branch ancestry.
- Sending main-only follow-up defects to `release/17.0.0` when the related fix is not on the release branch. Mark these contingent instead.
- Using labels alone. Labels are hints; issue evidence and the release tracker decide.
- Treating tracker `Mode: development` as "no release train." The branch can still exist; target selection still matters.
4 changes: 4 additions & 0 deletions .agents/skills/react-on-rails-release-train-issue-evaluation/agents/openai.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
interface:
display_name: "React on Rails Release Train Issue Evaluation"
short_description: "Classify issues for release train or main"
default_prompt: "Use $react-on-rails-release-train-issue-evaluation to decide whether recent React on Rails issues belong on release/17.0.0 or main."

@greptile-apps greptile-apps Bot Jul 9, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Version hardcoded in default_prompt — same release/17.0.0 specificity appears here and will need a manual update every time the active release train advances. Using the form release/X.Y.Z (or deferring to the skill's own language) would make this durable. The other two fields (display_name, short_description) are already version-neutral.

8 changes: 7 additions & 1 deletion AGENTS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,8 @@ React on Rails is a Ruby gem + npm package that integrates React with Ruby on Ra
workflow `SKILL.md` files installed in the user's or agent's normal skill
directory; duplicating them here creates duplicate Codex skill picker entries.
Current repo-specific skills include `$stress-test`,
`$optimize-rsc-performance`, and `$react-on-rails-update-changelog`.
`$optimize-rsc-performance`, `$react-on-rails-update-changelog`, and
`$react-on-rails-release-train-issue-evaluation`.
- `.claude/skills`: symlink to `.agents/skills` so Claude Code exposes the
repo-specific local skills kept by this checkout. Shared skills should come
from the installed shared pack, not this symlink.
Expand Down Expand Up @@ -58,6 +59,11 @@ React on Rails is a Ruby gem + npm package that integrates React with Ruby on Ra
- When deciding whether an issue or proposed fix is worth doing, use the
installed/shared `$evaluate-issue` skill; a short invocation is
`$evaluate-issue` or "Is this issue worth fixing?"
- When deciding whether a React on Rails issue belongs on the active release
train (`release/X.Y.Z`, especially `release/17.0.0`) or should wait for
`main`, use the repo-local
`.agents/skills/react-on-rails-release-train-issue-evaluation/SKILL.md`; a
short invocation is `$react-on-rails-release-train-issue-evaluation`.
- When the user wants a ready prompt for review-only GitHub issue triage or an
all-open-issues audit, use the installed/shared `$plan-issue-triage` skill; a
short invocation is `$plan-issue-triage` or "Plan an issue triage"
Expand Down
Loading