Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Shell scripts must check out LF everywhere: bash (incl. Git Bash on Windows) chokes on
# CRLF, so an autocrlf checkout would ship broken hooks and rails scripts.
*.sh text eol=lf
10 changes: 6 additions & 4 deletions kit/CLAUDE.md.template
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
{{ONE_PARAGRAPH: what the system does, who uses it, the business outcome it serves —
in the client's words.}}

**Stack:** {{e.g. .NET 8 / Angular / SQL Server / Azure / GitHub Actions}}
**Stack:** {{e.g. .NET 8 / Angular / SQL Server / Azure / GitHub Actions or Azure Pipelines}}
**Build / test / verify:** `{{BUILD_CMD}}` · `{{TEST_CMD}}` · `{{LINT_CMD}}`
**Done means the hook lets you stop.** A change is finished when it is proven against its
spec by something other than its author — not when the code exists. The blocking Stop hook
Expand Down Expand Up @@ -74,7 +74,8 @@ HIGH = all five rungs, including security-reviewer and a named human sign-off.
Any change touching these is held to extra review and may trip the security gate:
- `{{AUTH_PATHS}}` — authentication / identity / authorization
- `{{DATA_PATHS}}` — schema migrations, data access
- `{{PIPELINE_PATHS}}` — `.github/`, the hooks, this CLAUDE.md, branch protection
- `{{PIPELINE_PATHS}}` — `.github/` or `.azuredevops/`, the hooks, this CLAUDE.md, branch
protection / branch policies
- `{{INFRA_PATHS}}` — `infra/` (IaC is HIGH risk every time)
- `{{PROMPT_PATHS}}` — prompts, model selection, tool definitions (see "Agentic work" below)

Expand Down Expand Up @@ -117,8 +118,9 @@ The route is a **spike**: bounded, throwaway work whose deliverable is a written
inflate all of them.

## Data boundary & secrets (non-negotiable)
- Secrets live in {{CLIENT}}'s Key Vault and GitHub secrets. **Never in code, never in this file,
never in specs.** A key in a commit is the one unrecoverable mistake.
- Secrets live in {{CLIENT}}'s Key Vault and GitHub secrets / Azure DevOps variable groups
(Key-Vault-backed). **Never in code, never in this file, never in specs.** A key in a commit
is the one unrecoverable mistake.
- The {{LLM_PROVIDER}} key is {{CLIENT}}-procured under their own agreement.
- Tag genuinely sensitive material out of agent context. If the engagement handles regulated or
client-sensitive data, write `docs/data-flow-brief.md` during setup — a one-pager covering what
Expand Down
21 changes: 14 additions & 7 deletions kit/HARNESS.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ AI is doing the work. This page explains each piece in plain terms.
| `.claude/agents/` | Six AI specialists it can call — two arrive automatically |
| `.claude/skills/` | The team's runbooks for recurring jobs (specs, tests, bugs, PRs) |
| `.mcp.json` | Shared plug-ins that give the AI extra abilities — approve once, done |
| `.github/workflows/` | The pull-request checks that apply to *everyone*, human or AI |
| `.github/workflows/` (GitHub) / `.azuredevops/pipelines/` (Azure DevOps) | The pull-request checks that apply to *everyone*, human or AI |
| `specs/`, `.sdlc/`, `infra/` | Feature specs, project phase tracking, infrastructure starter |

## Two ideas explain the whole design
Expand Down Expand Up @@ -134,10 +134,12 @@ Personal plug-ins stay in your own machine's config — this file is only for wh
team should share. Tools that install themselves (like GitNexus) come with printed
instructions instead: `.claude/tools/<tool>/SETUP.md`.

## The pull-request checks — .github/workflows/
## The pull-request checks — .github/workflows/ or .azuredevops/pipelines/

These run at the server and apply to every change, no matter who or what wrote it. The
principle: **work can be proposed by anyone; only a gate lets it through.**
On GitHub these are Actions workflows in `.github/workflows/`; on Azure DevOps the same
checks are Azure Pipelines in `.azuredevops/pipelines/`. Either way they run at the server
and apply to every change, no matter who or what wrote it. The principle: **work can be
proposed by anyone; only a gate lets it through.**

| Check | Power | Why |
|---|---|---|
Expand All @@ -149,7 +151,8 @@ principle: **work can be proposed by anyone; only a gate lets it through.**
| Deploy to dev | ships | after a green merge; rolls itself back on failure |
| Eval checks | blocks / advises | for AI-powered features: blocks if quality measurably drops |

On top sits **branch protection**: the blocking checks are mandatory, and a person who didn't
On top sits **branch protection** (on Azure DevOps: **branch policies** — same idea, the
platform's own mechanism): the blocking checks are mandatory, and a person who didn't
write the change must approve it. Everything above in one sentence: *machines verify the
facts; a human makes the call.*

Expand All @@ -161,8 +164,12 @@ never seen catch anything is decoration.

- **`specs/`** — one spec per feature. The starting point of all build work.
- **`.sdlc/`** — which project phase we're in, plus the record of decisions and artifacts.
- **`.github/profile/rubrics/`** — the written standards the AI reviewers judge against.
- **`scripts/rails/`** — helper scripts the checks use (branch protection, change scoping).
- **`.github/profile/rubrics/`** (GitHub) / **`.azuredevops/rails/rubrics/`** (Azure DevOps) —
the written standards the AI reviewers judge against. On Azure DevOps the override and
accepted-risk ledgers live beside them in `.azuredevops/rails/`.
- **`scripts/rails/`** — helper scripts the checks use (change scoping, plus the one-time
branch setup: `apply-branch-protection.sh` on GitHub, `configure-branch-policies.sh` on
Azure DevOps).
- **`infra/`** — an infrastructure starting point. Adapt it; don't use it as-is.
- **`.claude/harness-manifest.json`** — the install receipt: which harness version this repo
has and a fingerprint of every file *as installed*. It's how an upgrade can tell "still
Expand Down
35 changes: 23 additions & 12 deletions kit/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,15 @@ Each piece is tagged so you adopt deliberately:
Where each kit file goes in the client repo. The workflows and hooks reference these exact paths,
so install here unless you also repoint the references.

The `.github/...` destinations are the GitHub-Actions layout. On an `azure-devops` profile the
platform-aware installer skips the pure-GitHub payload (`workflows/`, `profile/rulesets/`,
`profile/CODEOWNERS`, `apply-branch-protection.sh`) and overlays the azure-devops pack instead:
the pipelines land in `.azuredevops/pipelines/`, and the neutral governance content — the rubrics
and the two ledgers — is redirected to `.azuredevops/rails/`, where those pipelines read it.
**Branch policies** (`.azuredevops/rails/branch-policies.json`, applied by
`scripts/rails/configure-branch-policies.sh`) replace the ruleset and CODEOWNERS. `.github/RAILS.md`
keeps its path on both platforms — the ADO pack overlays its own realization there on purpose.

| Kit path | Install to | Notes |
|---|---|---|
| `CLAUDE.md.template` | `./CLAUDE.md` | Replace every `{{TOKEN}}`; delete guidance comments. |
Expand All @@ -60,18 +69,18 @@ so install here unless you also repoint the references.
| `hooks/*` | `./.claude/hooks/` | `stop-gate`, `review-gate`, `save-review-receipt` (`.ps1` + `.sh`), plus `sensitive-edit-nudge` — an advisory (non-blocking) example, installed but not registered. |
| `agents/*` | `./.claude/agents/` | `planner`, `architect`, `grader`, `security-reviewer`, `build-error-resolver`, `debugger` — model-tiered; see `agents/README.md`. |
| `skills/*` | `./.claude/skills/` | `spec-writer`, `test-writer`, `api-pattern`, `pr-writer`, `eval-builder`, `diagnose`. |
| `workflows/{ci,grader,correctness,security,deploy-dev,eval-regression,eval-suite}.yml` | `./.github/workflows/` | The five rails + the two eval workflows. |
| `workflows/deploy-promote.yml` | `./.github/workflows/` | The second half of the deploy rail: dev→test→prod. Manual trigger only — the target Environment's **required reviewers** are the human go/no-go, and the workflow refuses to run against an environment that has none. |
| `workflows/RAILS.md` | `./.github/RAILS.md` | Operator's guide + shakedown drills. |
| `profile/rubrics/*` | `./.github/profile/rubrics/` | Workflows read these by this path. |
| `workflows/dependency-scan.yml` | `./.github/workflows/` | Weekly scan of the **standing stock** of third-party advisories. Raises (and self-closes) one issue; never blocks — the blocking, diff-scoped half is `ci.yml`'s `dependency-gate` job. On by default: a security scan you have to remember to switch on is not running. |
| `workflows/rails-telemetry.yml` | `./.github/workflows/` | Weekly gate-outcome report, committed as `.github/rails-telemetry.json`. Records what ran, every override by name, and **which checks branch protection actually requires vs which gate jobs exist** — the comparison that catches a gate someone disarmed. No external calls: it reads this repo's own history and writes into this repo. |
| `profile/rails-telemetry.schema.json` | `./.github/rails-telemetry.schema.json` | The report's shape, fixed at version 1 before the install wave so it is not retrofitted across live repos. The fleet collector refuses a version it does not know rather than misreading it. |
| `profile/eval-bypasses.md` | `./.github/eval-bypasses.md` | Override/bypass ledger. |
| `profile/dependency-exceptions.md` | `./.github/dependency-exceptions.md` | Accepted-risk ledger for `dependency-gate`. The `accepted-risk:dependency` label clears one PR; this records why, who decided, whether the vulnerable path is reachable, and when the acceptance expires. Swept at Setup review. |
| `profile/CODEOWNERS` | `./.github/CODEOWNERS` | |
| `workflows/{ci,grader,correctness,security,deploy-dev,eval-regression,eval-suite}.yml` | `./.github/workflows/` (ADO: `./.azuredevops/pipelines/`) | The five rails + the two eval workflows. |
| `workflows/deploy-promote.yml` | `./.github/workflows/` (ADO: `./.azuredevops/pipelines/`) | The second half of the deploy rail: dev→test→prod. Manual trigger only — the target Environment's **required reviewers** are the human go/no-go, and the workflow refuses to run against an environment that has none. |
| `workflows/RAILS.md` | `./.github/RAILS.md` | Operator's guide + shakedown drills. Same path on ADO installs — the ADO pack overlays its own realization here on purpose. |
| `profile/rubrics/*` | `./.github/profile/rubrics/` (ADO: `./.azuredevops/rails/rubrics/`) | Workflows/pipelines read these by this path. |
| `workflows/dependency-scan.yml` | `./.github/workflows/` (ADO: `./.azuredevops/pipelines/`) | Weekly scan of the **standing stock** of third-party advisories. Raises (and self-closes) one issue; never blocks — the blocking, diff-scoped half is `ci.yml`'s `dependency-gate` job. On by default: a security scan you have to remember to switch on is not running. |
| `workflows/rails-telemetry.yml` | `./.github/workflows/` (ADO: `./.azuredevops/pipelines/`) | Weekly gate-outcome report, committed as `.github/rails-telemetry.json`. Records what ran, every override by name, and **which checks branch protection (ADO: branch policies) actually requires vs which gate jobs exist** — the comparison that catches a gate someone disarmed. No external calls: it reads this repo's own history and writes into this repo. |
| `profile/rails-telemetry.schema.json` | `./.github/rails-telemetry.schema.json` | The report's shape, fixed at version 1 before the install wave so it is not retrofitted across live repos. The fleet collector refuses a version it does not know rather than misreading it. Deliberately `.github/` on **both** platforms: both packs' telemetry pipelines commit the report to `.github/rails-telemetry.json` so one collector reads a mixed GitHub/Azure fleet. |
| `profile/eval-bypasses.md` | `./.github/eval-bypasses.md` (ADO: `./.azuredevops/rails/eval-bypasses.md`) | Override/bypass ledger. |
| `profile/dependency-exceptions.md` | `./.github/dependency-exceptions.md` (ADO: `./.azuredevops/rails/dependency-exceptions.md`) | Accepted-risk ledger for `dependency-gate`. The `accepted-risk:dependency` label clears one PR; this records why, who decided, whether the vulnerable path is reachable, and when the acceptance expires. Swept at Setup review. |
| `profile/CODEOWNERS` | `./.github/CODEOWNERS` | ADO: skipped — the required-reviewer branch policy in `branch-policies.json` is the twin. |
| `profile/scripts/*` | `./scripts/rails/` | Workflows call `scripts/rails/diff-anchors.sh`. |
| `profile/rulesets/branch-protection.json` | `./.github/rulesets/` | Copied on install; `scripts/rails/apply-branch-protection.sh` reads it from there and applies it to GitHub. |
| `profile/rulesets/branch-protection.json` | `./.github/rulesets/` | Copied on install; `scripts/rails/apply-branch-protection.sh` reads it from there and applies it to GitHub. ADO: skipped — `.azuredevops/rails/branch-policies.json`, applied by `scripts/rails/configure-branch-policies.sh`, is the twin. |
| `eval-datasets/*` | `./eval-datasets/` | Golden-set template + how-to (§11 work only). |
| `prompts/*` | `./prompts/` | Versioned judge prompts (§11 work only). |
| *(generated)* | `./.claude/harness-manifest.json` | Not a kit file — the installer writes it: plugin version + sha256 of every file as installed. `/sdlc-upgrade` reads it to tell factory-original files (safe to update) from repo-adapted ones (left alone). Commit it. |
Expand Down Expand Up @@ -103,7 +112,7 @@ have run) **and `/update-docs`** — declared there, not duplicated here.
Plus `eval-regression.yml` (per-PR gate when prompts/models/tools/agent-behavior change) and
`eval-suite.yml` (periodic full benchmark) for §11 agentic deliverables.

The merge bar (in `branch-protection.json`): **CI green + spec-gate green + grader ran +
The merge bar (in `branch-protection.json`; ADO: `branch-policies.json`): **CI green + spec-gate green + grader ran +
correctness passed (or recorded override) + a non-author approval**; HIGH adds security pass + a
named sign-off in the PR.

Expand All @@ -119,6 +128,8 @@ named sign-off in the PR.
4. **Workflows + profile** — set the `<<PLACEHOLDER>>` build/test/deploy commands and gated-path
regex. See `workflows/README.md`.
5. **Branch protection** — run `scripts/rails/apply-branch-protection.sh` once GitHub Actions is on.
On Azure DevOps: **branch policies** — run `scripts/rails/configure-branch-policies.sh` once the
pipelines exist.
6. **§11 only** — wire the eval runner behind the `eval-*` workflows; calibrate thresholds.

## Then prove the rails — do not assume them
Expand Down
21 changes: 14 additions & 7 deletions kit/hooks/review-gate.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,8 @@
# which settings.json permission patterns cannot express.
#
# Fires on the Claude Code `PreToolUse` event for Bash. Before the agent is allowed to
# `git push` or `gh pr create` a change that touches compilable source, this proves that
# `git push`, `gh pr create`, or `az repos pr create` a change that touches compilable
# source, this proves that
# `/code-review` AND `/simplify` were actually run against the EXACT commit being pushed —
# not the agent's recollection that it did them. A missing receipt blocks the push and
# tells the agent to run the reviews.
Expand Down Expand Up @@ -69,10 +70,10 @@ $payload = $null
if ($raw) { try { $payload = $raw | ConvertFrom-Json } catch { } }
if (-not $payload) { Allow }

# --- Only gate Bash commands that actually INVOKE git push / gh pr create. Match the
# start of each shell segment (split on ; && || | &) so the words appearing inside a
# quoted string, an echo, or `--help` text don't trip the gate. `git -C <path> push`
# is recognized.
# --- Only gate Bash commands that actually INVOKE git push / gh pr create /
# az repos pr create. Match the start of each shell segment (split on ; && || | &) so
# the words appearing inside a quoted string, an echo, or `--help` text don't trip the
# gate. `git -C <path> push` is recognized.
if ($payload.tool_name -ne 'Bash') { Allow }
$cmd = [string]$payload.tool_input.command
if (-not $cmd) { Allow }
Expand All @@ -87,9 +88,15 @@ if ($cmd -match $pipeToShellRe) {
}

$gates = $false
foreach ($seg in ($cmd -split '\|\||&&|[;|&]')) {
# Split on newlines too — a multi-line Bash command with the trigger on a later line must not
# walk past this twin while the .sh form (which splits per input line via awk) denies it.
foreach ($seg in ($cmd -split '\|\||&&|[;|&]|\r?\n')) {
$s = $seg.Trim()
if ($s -match '^git\s+(-C\s+\S+\s+)?push(\s|$)' -or $s -match '^gh\s+pr\s+create(\s|$)') {
# Strip leading VAR=value assignments (see the .sh twin): an env-prefixed invocation like
# `AZURE_DEVOPS_EXT_PAT=... az repos pr create` is still an invocation.
while ($s -match '^[A-Za-z_][A-Za-z_0-9]*=\S*\s+') { $s = $s.Substring($Matches[0].Length) }
if ($s -match '^git\s+(-C\s+\S+\s+)?push(\s|$)' -or $s -match '^gh\s+pr\s+create(\s|$)' -or
$s -match '^az\s+repos\s+pr\s+create(\s|$)') {
$gates = $true; break
}
}
Expand Down
16 changes: 12 additions & 4 deletions kit/hooks/review-gate.sh
Original file line number Diff line number Diff line change
Expand Up @@ -49,17 +49,25 @@ if [[ "$cmd" =~ $pipe_to_shell_re ]]; then
deny "Blocked: piping a download straight into a shell (curl|sh and friends) is denied by policy. Download to a file, inspect it, then execute it as a separate, reviewable step."
fi

# --- Only gate commands that actually INVOKE git push / gh pr create. Inspect the start
# of each shell segment (split on ; && || | &) so the words inside a quoted string or
# an echo don't trip the gate. `git -C <path> push` is recognized.
# --- Only gate commands that actually INVOKE git push / gh pr create / az repos pr create.
# Inspect the start of each shell segment (split on ; && || | &) so the words inside a
# quoted string or an echo don't trip the gate. `git -C <path> push` is recognized.
# awk, not sed: BSD sed renders '\n' in a replacement as a literal 'n', which would
# leave compound commands unsplit and let `cd x && git push` walk past the gate.
gates=false
segment_stream="$(printf '%s' "$cmd" | awk '{gsub(/\|\||&&|[;|&]/, "\n"); print}')"
while IFS= read -r seg; do
s="$(printf '%s' "$seg" | sed -E 's/^[[:space:]]+//')"
# Strip leading VAR=value assignments: `AZURE_DEVOPS_EXT_PAT=... az repos pr create` is a
# documented az auth pattern and still an invocation — the shell treats the prefix as
# environment, not the command. (A value with quoted whitespace defeats this; rare, and
# under-gating there matches the parser's documented coverage boundary.)
while [[ "$s" =~ ^[A-Za-z_][A-Za-z_0-9]*=[^[:space:]]*[[:space:]]+ ]]; do
s="${s:${#BASH_REMATCH[0]}}"
done
if [[ "$s" =~ ^git[[:space:]]+(-C[[:space:]]+[^[:space:]]+[[:space:]]+)?push([[:space:]]|$) ]] \
|| [[ "$s" =~ ^gh[[:space:]]+pr[[:space:]]+create([[:space:]]|$) ]]; then
|| [[ "$s" =~ ^gh[[:space:]]+pr[[:space:]]+create([[:space:]]|$) ]] \
|| [[ "$s" =~ ^az[[:space:]]+repos[[:space:]]+pr[[:space:]]+create([[:space:]]|$) ]]; then
gates=true
break
fi
Expand Down
3 changes: 3 additions & 0 deletions kit/packs/cicd/azure-devops/pack.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,8 @@ provides:
- scripts/run-claude-review.sh # headless Claude Code CLI invocation (no action exists on ADO)
- scripts/post-pr-thread.sh # POST/PATCH a PR thread via the ADO REST API
branch_policies: branch-policies/policies.json # desired-state consumed by configure-branch-policies.sh
settings_fragment: settings.fragment.json # az tooling perms + .azuredevops/ gated paths,
# merged into .claude/settings.json
rails_guide: RAILS.md # operator's guide + shakedown drills (ADO realization)
mcp_fragment: mcp.fragment.json # Azure DevOps MCP server, merged into .mcp.json
# (ADO_ORGANIZATION_NOT_SET is the Phase-3 placeholder —
Expand Down Expand Up @@ -116,6 +118,7 @@ overlays:
# Overlays the core's GitHub-flavored guide (last wins) so the documented path
# (.github/RAILS.md, referenced by /sdlc-setup Step 8) holds the ADO realization.
- { src: RAILS.md, dest: .github/RAILS.md }
- { src: settings.fragment.json, dest: .claude/settings.json, merge: true }
- { src: mcp.fragment.json, dest: .mcp.json, merge: true }
# Installed by the core/profile layer, referenced here (NOT shipped by this pack — see `consumes`):
# scripts/rails/diff-anchors.sh <- kit/profile/scripts/diff-anchors.sh
Expand Down
Loading