Add the decisions skill (evidence-cited decision report)#250
Open
jakelevirne wants to merge 1 commit into
Open
Add the decisions skill (evidence-cited decision report)#250jakelevirne wants to merge 1 commit into
jakelevirne wants to merge 1 commit into
Conversation
Mines SpecStory histories into an evidence-cited decision log: decisions made, decisions changed (with supersession chains per entity), open decisions, and the entities decided about - plus insights: churn hotspots (2+ reversals), provisional "for now" debt never revisited, and re-litigated decisions. Deterministic high-recall engine (signal grammars, entity clustering with a ubiquity cap, prompt-text dedupe, no LLM/network) + agent precision pass via SKILL.md (drop false positives, resolve referents from evidence refs, name entities, write the ADR-style report). Fully standalone: own engine copy, own corpus (~/.specstory/decisions.db), own installer. 9 tests over fixtures covering every lifecycle; exercised against the real corpus.
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a new standalone decisions skill that indexes .specstory/history transcripts into a local SQLite corpus and deterministically mines an evidence-cited decision digest/JSON (decided/changed/open + insights like churn/provisional/reopened), with an install script and fixtures-backed tests.
Changes:
- Introduces a Node (ESM, zero-deps) engine: transcript parsing (
patterns.mjs/parse.mjs), discovery/indexing (discover.mjs/indexer.mjs/db.mjs), and decision mining/rendering (decide.mjs) exposed via a CLI (scripts/decisions.mjs). - Adds the
decisionsskill definition (SKILL.md), install helper (install.sh), and user docs (README.md,package.json). - Adds engine tests and fixture corpora to validate decision lifecycle, supersession chains, insights, and windowing.
3 most important improvements:
- Make evidence refs stable and truly deterministic by removing
process.cwd()-relative evidence paths incomputeDecisions()(critical correctness + usability). - Fix the
DECIDE_REtypo that prevents matching common “don’t use X” decisions. - De-flake tests by ensuring each test run uses a unique SQLite DB filename (avoid concurrent clobbering).
Tests to consider adding/updating:
- Add a fixture/test that asserts “don’t use Postgres” is detected as a firm decision (covers the
DECIDE_REbranch). - Add a test that asserts evidence paths don’t change when running from different working directories (guards the determinism guarantee).
Reviewed changes
Copilot reviewed 18 out of 18 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| decisions/tests/decisions.test.mjs | Node test suite validating decision lifecycle, insights, determinism, and windowing. |
| decisions/SKILL.md | Skill instructions for indexing/mining decisions and producing the final report. |
| decisions/scripts/lib/patterns.mjs | Transcript format grammar, tokenization/outcome classifiers, and secret redaction. |
| decisions/scripts/lib/parse.mjs | Pure transcript parser emitting “beats” with tool/cmd/file metadata. |
| decisions/scripts/lib/indexer.mjs | Incremental indexing into SQLite with idempotency contract and pruning helpers. |
| decisions/scripts/lib/discover.mjs | Project/history discovery and stable project identity resolution. |
| decisions/scripts/lib/decide.mjs | Decision mining, clustering, supersession/status, insights, and digest rendering. |
| decisions/scripts/lib/db.mjs | SQLite schema, migrations, and session child-row cleanup helpers. |
| decisions/scripts/decisions.mjs | CLI entrypoint: index + decisions/report (digest/JSON/out). |
| decisions/README.md | User-facing documentation: what it is, install/use, sample output, dev workflow. |
| decisions/package.json | Package metadata and Node test script. |
| decisions/install.sh | Installer bundling the engine + skill into the user’s skills directory. |
| decisions/fixtures/decisions-beta/.specstory/history/2026-06-20_09-00-00Z-flagparser-kebab.md | Fixture transcript (decision). |
| decisions/fixtures/decisions-beta/.specstory/history/2026-06-18_09-00-00Z-settingspanel-question.md | Fixture transcript (open question). |
| decisions/fixtures/decisions-alpha/.specstory/history/2026-06-15_09-00-00Z-eventstore-duckdb.md | Fixture transcript (decision change). |
| decisions/fixtures/decisions-alpha/.specstory/history/2026-06-10_09-00-00Z-authclient-fornow.md | Fixture transcript (provisional decision). |
| decisions/fixtures/decisions-alpha/.specstory/history/2026-06-09_09-00-00Z-eventstore-sqlite.md | Fixture transcript (decision change). |
| decisions/fixtures/decisions-alpha/.specstory/history/2026-06-02_09-00-00Z-eventstore-postgres.md | Fixture transcript (initial decision). |
Comment on lines
+78
to
+96
| const cutoff = days > 0 ? new Date(now - days * 86400000).toISOString().slice(0, 10) : null | ||
| const cwd = process.cwd() | ||
|
|
||
| // 1) Extract candidates: one per (beat, matching sentence). | ||
| const cands = [] | ||
| for (const r of rows) { | ||
| if (cutoff && r.date < cutoff) continue | ||
| for (const sent of sentences(r.intent)) { | ||
| const change = CHANGE_RE.test(sent) | ||
| const open = OPEN_RE.test(sent) | ||
| const decide = DECIDE_RE.test(sent) | ||
| const provisional = PROVISIONAL_RE.test(sent) | ||
| if (!change && !open && !decide && !provisional) continue | ||
| // precedence: an explicit question is open even if it contains decision verbs | ||
| const kind = change ? 'change' : (open ? 'open' : (decide ? 'decide' : 'provisional')) | ||
| cands.push({ | ||
| pid: r.pid, project: r.pname, sid: r.sid, date: r.date, ord: r.ord, | ||
| evidence: `${relative(cwd, r.path) || r.path}:${r.line}`, | ||
| quote: redactSecrets(clip(sent)), |
| // --- signal grammars (ordered by precedence: CHANGE > OPEN > DECIDE > PROVISIONAL) --- | ||
| // A change is itself a decision (it decides the new thing) AND marks its predecessors. | ||
| const CHANGE_RE = /\b(?:actually,?\s+(?:let'?s|we|use|go|switch|make)|instead of what|changed?\s+(?:my|our)\s+mind|scrap\s+that|forget\s+that|on\s+second\s+thought|let'?s\s+not\b|go\s+back\s+to|switch(?:ing)?\s+(?:\S+\s+){0,3}?(?:back\s+)?to\b|revert\s+to|no\s+longer\s+(?:use|using)|replace\s+\S+\s+with)/i | ||
| const DECIDE_RE = /\b(?:let'?s\s+(?:go\s+with|use|stick\s+with|keep|standardize\s+on|make|call|do)|go(?:ing)?\s+with|we(?:'ll|\s+will)\s+(?:use|go|keep|stick)|decided?\s+(?:on|to|that)|decision\s*:|settle[d]?\s+on|agreed?\s+(?:on|to)\b|standardize\s+on|default\s+to|always\s+use|never\s+use|don'?t\s+use\s+(?:\S+\s+){0,4}?use|rename\s+(?:\S+\s+){0,4}?to\b|call\s+it\s+\S|(?:option|choice)\s+\d|the\s+(?:first|second|third)\s+option|use\s+(?:\S+\s+){0,4}?(?:instead\s+of|rather\s+than))/i |
| const NOW = Date.parse('2026-06-24T12:00:00Z') | ||
|
|
||
| function build(days = 0) { | ||
| const db = openDb(join(tmpdir(), `decisions-test-${process.pid}-${days}.db`)) |
Comment on lines
+10
to
+16
| // STATUS MODEL (one per decision): | ||
| // decided - a firm choice, still standing (the latest firm decision on its entity) | ||
| // changed - superseded by a later firm decision on the same entity (chained) | ||
| // open - raised but unresolved: a question, or provisional ("for now") with no | ||
| // later firm decision on the entity | ||
| // Flags: provisional ("for now"/TBD language), reopened (a later open question on an | ||
| // entity that already had a firm decision). |
| @@ -0,0 +1,76 @@ | |||
| // db.mjs - the lore corpus schema (SQLite via node:sqlite, Node >= 22.5; zero deps). | |||
Comment on lines
+50
to
+52
| // ---------- the runs journal: Lore's memory of its own activity ---------- | ||
| // One row per notable engine invocation (auto) plus one per /lore run (the agent's Step 6 duty). | ||
| // This is what makes "what has Lore done?" a query instead of archaeology. |
Comment on lines
+180
to
+181
| // here, before display, so the agent never has to handle a live credential; the agent-side | ||
| // scrub at forge time (SKILL.md Step 5) remains as defense in depth. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds decisions, a standalone, separately-installable skill that mines SpecStory
histories into an evidence-cited decision report (a lightweight, auto-mined ADR log):
(e.g. Postgres -> SQLite -> DuckDB)
Insights: churn hotspots (entities re-decided 2+ times), provisional "for now" debt
never revisited, re-litigated decisions (settled, then re-questioned), plus agent-flagged
decisions lacking rationale.
Purely additive vs
dev- one new top-leveldecisions/directory. Independent oflore/andworkthreads/(own engine copy, own corpus~/.specstory/decisions.db,own installer).
How it works
Deterministic high-recall engine (signal grammars over user turns, entity clustering
with a ubiquity cap, prompt-text dedupe, supersession from timeline order; byte-identical
output, no LLM/network) + an agent precision pass via
SKILL.md(drop false positives,resolve referents like "I like option 1" by reading the cited evidence, name entities,
write the report to
.specstory/decisions/<date>-decisions.md).Verification
fixed in-engine (duplicate prompt text, "placeholder" false positives, ubiquitous-entity
chaining)
decisions/install.shand verified from an unrelated project directoryMerge
Intended to land as a single commit via Squash and merge.
🤖 Generated with Claude Code