Skip to content

Issue triage with Jev: categories, complexity 1–10, autonomous severity check, and dashboard summary cards - #3

Merged
Kadreev merged 26 commits into
mainfrom
feat/issue-triage-jev
Oct 5, 2026
Merged

Kadreev merged 26 commits into
mainfrom
feat/issue-triage-jev

Conversation

@Kadreev

@Kadreev Kadreev commented Oct 4, 2026

Copy link
Copy Markdown
Owner

What changed

Issue triage with TypeSafe's Jev, plus summary cards for the dashboard. It implements the approved spec docs/superpowers/specs/2026-10-04-issue-triage-jev-design.md (revision 2.1) through the 12-task plan in docs/superpowers/plans/.

Triage data

  • New optional issue fields:
    • category: one of 11 defaults, or a project's own list in qa-tracker.config.json. Each category maps to type.
    • complexity: 1–10, meaning effort to fix.
    • details: free-text description.
  • severity is now optional.
  • Per-field provenance in issue.triage records whether Jev set the value or someone set it on purpose. A new assessments.yaml keeps every Jev judgment and is only ever appended to.
  • validate now separates errors from warnings. Changing the category list, or a stale entry in the history file, never blocks a write. validate() keeps its old contract, and validateAll() returns both.

Autonomous triage, with no human accept step

  • qa-tracker assess asks Jev for category, complexity and severity. Code-side confidence gates (70% / 50% / 85%) decide what gets applied.
    • Jev only fills empty fields and refreshes values it set earlier. It never overwrites a value someone set on purpose.
    • Uncertain answers, and confident disagreements with explicit values, go to a triage queue (qa-tracker triage). Agents resolve the queue with set.
  • --dry-run prints the exact requests and needs no key. Auto-assessing new issues on add-issue is opt-in: "jev": { "auto_assess": true }, or --assess.
  • The client uses Node's built-in fetch (no new dependency), with retries for 429/529, network errors and timeouts. A 401 stops the run.

Views

  • STATUS.md and the dashboard get Category/Cx columns, the ᴶ marker on Jev-set values, a severity vs complexity vs status legend, and the triage queue.
  • The dashboard gets edit-mode selects for category, complexity and severity.
  • Summary cards (Open issues, Fixed, Coverage, Hotspot) and panels (Issues by feature, Severity × status, Work queue) sit above the matrix. Table legends become header tooltips, with a print-only legend for PDF export. STATUS.md gets a matching Summary block.

Why

Findings needed finer categories than the three types, and a fix-effort estimate next to impact. Severity (impact) and status (lifecycle) were being confused. The user also wanted this to run without anyone in the loop, so agents stay autonomous: code applies Jev's judgments, and agents handle whatever the code won't apply.

For reviewers

  • Privacy: only assess, and opted-in add-issue, send anything. They send the issue title and details, the feature's name, area and routes, linked surfaces' text, and the question text including category descriptions. Recorded labels, status and file contents are never sent. The key is read only from TYPESAFE_API_KEY, never written, printed or included in errors. The dashboard server makes no outbound requests.
  • Backward compatibility: a 0.2.0 tracker validates cleanly with no warnings. assessments.yaml is created only on the first assess.
  • Release: this is additive, so it should ship as 0.3.0. Entries are under CHANGELOG [Unreleased], and package.json is unchanged until release.
  • Known follow-ups (not blocking):
    • An explicitly set category that isn't in the list gets re-sent on every default assess, which costs tokens.
    • --assess=constructor and similar prototype-key values pass the boolean lookup. Fix with Object.hasOwn.
    • Small duplicated code (isMapping and others).
    • Minor CSS edge cases at very narrow widths.
  • How it was built: subagent-driven, with a spec-and-quality review gate after every task and a final whole-branch review. Every finding from those reviews has been addressed except the deferred follow-ups above.
  • Live API: the request and response contract was checked against the TypeSafe docs and one real response. No test contacts the API. npm run smoke:jev is a manual live check that runs only when a key is set.

Testing

  • npm run check: 178/178 tests (up from 60 on v0.2.0), and the demo validates with no warnings.
  • The demo includes an example assessment (model: jev-example). Its triage queue shows one needs-triage and one disagrees item.
  • Browser-checked: dashboard in dark and light mode, header tooltips, the edit-mode selects, and phone width (375 px, no horizontal scroll).

🤖 Generated with Claude Code

Kadreev and others added 25 commits October 4, 2026 18:07
Categories (default list, replaceable per project), complexity 1-10 as fix
effort, and an independent Jev severity check, stored as suggestions in an
append-only assessments.yaml and applied only through `accept`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
No human accept step: rules in code apply confident judgments to gaps and
to Jev's own earlier values, never over explicit ones; uncertain answers and
confident disagreements go to a triage queue that agents resolve with set.
Also folds in the Fable review: non-fatal category drift, an append-only log
that can't fail validation, render-time disagreement, argmax complexity,
file-order ties, injectable backoff and UTC ids.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- a hand-edited Jev value is explicit and never overwritten
- type is derived from category and carries no provenance
- validate() keeps returning errors; validateAll() adds warnings
- auto-assess on add-issue is opt-in (jev.auto_assess or --assess)
- disagreements don't resurrect for an unchanged suggestion
- default assess only fills gaps; --refresh revises Jev's own values
- severity gate starts at 0.85; one number format; unset severity sorts last

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Eleven TDD tasks from the approved spec (revision 2.1): categories and
config, schema and validation, triage policy, edits and the assessment
commit, request builder, client, assess, agent CLI, STATUS.md, dashboard,
docs and demo.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…venance

Adds the optional issue fields details, category, complexity and triage, the
assessments.yaml log shape, and validateAll/store.check with warnings that never
block a write. severity is no longer required.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A gap whose latest answer passes the gate (value deleted afterwards, or the
category listed afterwards) no longer reports a misleading low confidence.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Approved in conversation: four headline cards (open, fixed, coverage,
hotspot), three panels (by feature, severity x status, work queue), the
same numbers as a STATUS.md Summary, and table legends turned into
accessible header tooltips with a print-only legend for PDF export.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The API returns `answers` keyed by question id (docs.typesafe.ai/api.md),
which mapAnswers consumes; the client rejected every real response as
malformed because it required an array.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add `triage [--json]`; `set <issue> category|complexity|severity|details`;
add-issue takes --details/--category/--complexity, optional --severity and
opt-in auto-assess (--assess/--no-assess, jev.auto_assess); `get` shows
triage provenance, the latest assessment and warnings naming the issue;
`validate` prints warnings. Pure formatters live in src/triage-format.mjs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Issue tables show Category and Cx with a J marker for unconfirmed Jev
values, a severity/complexity/status legend, and a Triage queue section
with the issue warnings under it. status and STATUS.md share one render.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Issue tables gain Category and Cx columns (unset shows an em dash, severity
class "sev none"), Jev-set cells carry a "Jev 91%" tooltip, edit mode gets
selects for category, complexity and severity, and a Triage queue section
mirrors STATUS.md. Issue sorting moves to shared helpers in format.mjs, which
also fixes the NaN ordering the dashboard had for unset severity.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The legend explains the mark, so Jev-owned cells now show it after the value,
as STATUS.md does, alongside the existing confidence tooltip.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
README "Triage with Jev" section, agent guide and skill without required
--severity, demo issues with categories/complexity and an illustrative
assessments.yaml, and the live smoke script (npm run smoke:jev).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
--assess=false used to store the string 'false' and opted in to sending the
issue to TypeSafe. Boolean flags now take true/1/false/0 after =, and anything
else is a usage error.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A 422 now reads 'request rejected (422): <API message>', and the 'shorten
details' hint is added only when the API message is about size, length,
tokens or being too large. Test fakes in the client, assess and CLI tests are
real Response objects, so the production text() + JSON.parse path is what runs;
the unused res.json() fallback in the client is removed, and an empty JSON
error body is no longer shown as '{}'.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…hanges in the changelog

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 4, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-04T20:20:55.235050Z f04db77 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f04db77f80

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/triage-policy.mjs Resolved
Comment thread src/assess.mjs Resolved
Comment thread src/assess.mjs Resolved
From the Codex review of #3:
- reject a confidence outside [0, 1] while mapping each answer, so one bad
  response fails its own issue instead of sinking the whole batch at validate
- an answer from a different model than the run's first fails that issue,
  so the single `model` on the assessment entry stays true for every judgment
- id-keyed result maps have no prototype (and add-assessment builds `applied`
  with Object.fromEntries), so an issue id of `__proto__` is committed like
  any other instead of silently vanishing

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Kadreev
Kadreev merged commit ce6a41b into main Oct 5, 2026
6 checks passed
@Kadreev
Kadreev deleted the feat/issue-triage-jev branch October 31, 2026 05:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant