Skip to content

feat(triage): add effort-based gating before auto-promoting to coder #36

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
rh-hemartin wants to merge 2 commits into
base: main
Choose a base branch
from
+130 −33
Open

feat(triage): add effort-based gating before auto-promoting to coder #36

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
99ca0f6
feat(triage): add effort-based gating before auto-promoting to coder
rh-hemartin Jul 7, 2026
8408ffc
fix(triage): address PR review feedback on effort gating
rh-hemartin Jul 7, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 20 additions & 1 deletion agents/triage.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -246,6 +246,24 @@ Information is sufficient for a developer to investigate and fix.

**Choosing a category:** the `feature` category covers issues that describe desired new behavior rather than a defect in existing functionality — the reporter expects something that has never been implemented. Use `feature` only when the described behavior clearly never existed in the product. If there is _any_ possibility the behavior is a regression (it used to work, or the reporter references a specific version where it worked), use `insufficient` instead and ask for version or timeline information. When in doubt, ask — do not prematurely reclassify.

**Estimating effort:** Estimate implementation effort on a 0.25–3 scale matching the RICE effort dimension used by the prioritize agent. Consider:

- **Files and lines of code:** Trivial one-liner vs multi-file change
- **Architectural impact:** Isolated bug fix vs requires changes across layers or modules
- **Testing complexity:** Simple unit test vs needs integration tests, mocking, or environment setup
- **Domain knowledge:** Requires deep understanding of subsystems or external dependencies

| Score | Meaning |
|-------|---------|
| 0.25 | Trivial — typo, config change, one-liner |
| 0.5 | Simple — small, well-scoped change |
| 1 | Medium — requires understanding context, touches a few files |
| 1.5 | Moderate — multiple components or some design work |
| 2 | Complex — significant implementation, testing, or coordination |
| 3 | Very complex — large scope, architectural changes, high risk |

Higher effort issues (effort >= 2.0) get routed to human review instead of auto-promoting to the coder — similar to how feature issues already require human prioritization. When in doubt between two scores, round up (bias toward human review). Only estimate effort for bug/docs/performance categories; feature issues already gate on human review.

```json
{
"action": "sufficient",
Expand All @@ -267,7 +285,8 @@ Information is sufficient for a developer to investigate and fix.
"environment": "Relevant environment details",
"impact": "Who is affected and how",
"recommended_fix": "What a developer should investigate.",
"proposed_test_case": "Conceptual description of a test that would verify the fix — what to test, expected vs actual behavior, and edge cases to cover. Do not assume a specific test framework or file layout."
"proposed_test_case": "Conceptual description of a test that would verify the fix — what to test, expected vs actual behavior, and edge cases to cover. Do not assume a specific test framework or file layout.",
"effort": 1.0
},
"comment": "A triage summary comment formatted in markdown, presenting the assessment to the maintainers. Include the proposed test case as a fenced code block.",
"label_actions": {
Expand Down
2 changes: 1 addition & 1 deletion docs/code.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ on issues (not PRs). The code agent is also triggered automatically when the

| Label | Meaning |
|-------|---------|
| `ready-to-code` | Triggers the code agent. Applied by the [triage](triage.md) post-script for low-risk categories (bug, documentation, performance), or manually by a human for feature work after prioritization. |
| `ready-to-code` | Triggers the code agent. Applied by the [triage](triage.md) post-script for low-risk, low-effort (< 2.0) categories (bug, documentation, performance), or manually by a human for feature work or high-effort issues after prioritization. |
| `ready-for-review` | Applied by the code agent's post-script after pushing a PR. In per-repo installs, triggers review when applied to a PR; also marks workflow state for humans and the retro agent. |

## Configuration and extension
Expand Down
4 changes: 2 additions & 2 deletions docs/triage.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,8 @@ outcome and the post-script applies the corresponding label.
| Label | Meaning |
|-------|---------|
| `needs-info` | The issue lacks sufficient information. The agent posted clarifying questions. |
| `ready-to-code` | The issue is fully specified and low-risk (bug, documentation, performance). Triggers the [code agent](code.md). |
| `triaged` | The issue is fully specified but is a feature or other category that requires human prioritization before coding. |
| `ready-to-code` | The issue is fully specified, low-risk (bug, documentation, performance), and low-effort (< 2.0). Triggers the [code agent](code.md). |
| `triaged` | The issue requires human prioritization before coding: feature work, other categories, or high-effort (>= 2.0) bug/docs/performance issues. |
| `duplicate` | The issue duplicates an existing one. The agent identified the original and the post-script closes the issue. |
| `blocked` | The issue depends on another issue or external condition. The agent identified the blocker. |

Expand Down
29 changes: 28 additions & 1 deletion schemas/triage-result.schema.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,27 @@
"if": { "properties": { "action": { "const": "sufficient" } }, "required": ["action"] },
"then": { "required": ["clarity_scores", "triage_summary"] }
},
{
"if": {
"required": ["action", "triage_summary"],
"properties": {
"action": { "const": "sufficient" },
"triage_summary": {
"required": ["category"],
"properties": {
"category": { "enum": ["bug", "documentation", "performance"] }
}
}
}
},
"then": {
"properties": {
"triage_summary": {
"required": ["effort"]
}
}
}
},

@waynesun09 waynesun09 Jul 10, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MEDIUM] Schema then clause relies on sibling allOf entry — fragile coupling

This then clause uses properties.triage_summary.required: ["effort"] but doesn't itself declare required: ["triage_summary"] at the root. It works today because allOf[2] already requires triage_summary when action=sufficient. But if someone reorders or removes that entry, this conditional silently stops enforcing anything.

Suggested fix — add defensive required at the then root:

"then": {
  "required": ["triage_summary"],
  "properties": {
    "triage_summary": {
      "required": ["effort"]
    }
  }
}

{
"if": { "properties": { "action": { "const": "prerequisites" } }, "required": ["action"] },
"then": {
Expand Down Expand Up @@ -132,7 +153,13 @@
"environment": { "type": "string" },
"impact": { "type": "string", "minLength": 1 },
"recommended_fix": { "type": "string", "minLength": 1 },
"proposed_test_case": { "type": "string", "minLength": 1 }
"proposed_test_case": { "type": "string", "minLength": 1 },
"effort": {

@waynesun09 waynesun09 Jul 8, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MEDIUM] effort is optional while the post-script treats it as load-bearing — silent-failure risk

effort isn't in triage_summary's required array, and no conditional (if/then) requires it for bug/docs/performance categories the way other conditionals in this schema do elsewhere. Since the harness's validation_loop only validates against this schema, a model that silently stops emitting effort would never fail validation — every bug/docs/performance issue would quietly start routing to triaged in post-triage.sh with no alarm raised anywhere.

(Also flagged by the qodo-code-review bot on this PR as "Missing effort blocks promotion".)

Suggested fix: add a conditional schema requirement for effort on bug/docs/performance categories when action: sufficient, or default missing effort to a low value in the script to match the prompt's "bias toward auto-promotion" guidance.

"type": "number",
"minimum": 0.25,
"maximum": 3,
"description": "Estimated implementation effort (0.25 = trivial, 1 = moderate, 2 = substantial, 3 = large)"
}
},
"additionalProperties": false
},
Expand Down
Loading
Loading