ci: add template validation workflow (Ajv + upstream schema)#1
Merged
mrkhachaturov merged 1 commit intomainfrom Apr 15, 2026
Merged
ci: add template validation workflow (Ajv + upstream schema)#1mrkhachaturov merged 1 commit intomainfrom
mrkhachaturov merged 1 commit intomainfrom
Conversation
Adds .github/workflows/validate.yml running on every PR and push to
main. Mirrors the upstream hindsight-docs/scripts/check-templates.mjs
pattern that PR #1065 wired into vectorize-io/hindsight's CI — fetch
bank-template-schema.json from upstream main, compile with Ajv, validate
every manifest referenced from templates.json.
Three validation layers:
1. Schema validation — Ajv compiles the upstream-generated JSON Schema
(authoritative: regenerated from BankTemplateManifest.model_json_schema()
on every upstream PR via verify-generated-files). Every
templates/*.json is validated against it. Catches field type
mismatches, unknown fields, missing required keys, invalid
MentalModelTrigger / TagGroup shapes, and everything else the upstream
Pydantic model enforces at parse time.
2. Catalog integrity — templates.json must have a templates array, every
entry needs id + manifest_file, IDs unique within the catalog, every
manifest_file ref points at an existing file, no orphan files under
templates/ (drop-and-forget regression guard).
3. HindClaw content guard — retain_extraction_mode must be in
{concise, verbose, custom, chunks}. Upstream's schema accepts any
string here but the runtime retain pipeline only honours those four
values; verbatim was dropped during the convergence refactor.
Files:
- .github/workflows/validate.yml — single job, ~30 lines, node 22 +
npm ci + curl the schema + node scripts/check-templates.mjs
- scripts/check-templates.mjs — adapted from upstream's check-templates.mjs
for this repo's flat layout (templates.json + templates/ at repo root
instead of hindsight-docs/src/data/). Adds the orphan detection and
content guard layers upstream doesn't need.
- package.json + package-lock.json — ajv ^8.17.1 as the sole dev dep,
"private": true so the package is not publishable
- .gitignore — /node_modules/ and /bank-template-schema.json (fetched
fresh at CI time, never committed)
Verified locally against all three committed templates (backend-python,
fullstack-typescript, astromech-test): all pass. Sanity-checked negative
paths by planting a schema violation (disposition_empathy = "five"),
the verbatim content guard, and an orphan file — all three are caught
with clear per-error output pointing at the failing path.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a CI workflow that validates every bank template in this repo against the upstream HindSight schema on every PR and push. Before this, a contributor could merge a structurally broken template and nothing would catch it until HindClaw tried to import it at runtime.
Mirrors the upstream
check-templates.mjspattern from vectorize-io/hindsight#1065, adapted for this repo's flat layout.Validation layers
bank-template-schema.jsonand validates eachtemplates/*.jsonagainst it. Authoritative: upstream regenerates the schema fromBankTemplateManifest.model_json_schema()on every PR viaverify-generated-files, so it always matches the live Pydantic model.templates.jsonhas atemplatesarray, every entry hasid+manifest_file, IDs unique within catalog, everymanifest_fileref points at an existing file, no orphan files undertemplates/.retain_extraction_mode∈ {concise,verbose,custom,chunks}. Upstream's schema accepts any string; the runtime retain pipeline only honours those four values.Files
.github/workflows/validate.ymlnode scripts/check-templates.mjsscripts/check-templates.mjspackage.json/package-lock.jsonajv ^8.17.1as the sole dev dep,"private": true.gitignore/node_modules/and/bank-template-schema.json(fetched fresh each CI run)Verified locally
All three committed templates (
backend-python,fullstack-typescript,astromech-test) pass. Sanity-checked the negative paths:disposition_empathy = "five"→ caught withmust be integerat/bank/disposition_empathy✓retain_extraction_mode = "verbatim"→ caught with valid-values listing ✓templates/→ caught with explicit error ✓Test plan
curlfetchesbank-template-schema.jsonfrom upstream main successfullynpm ciresolves from the committed lockfileNot included (deliberate)
schemas/bank-template-schema.json+ a cron job that PRs updates on drift.hindsight-api-sliminstall, and the runtime-only checks (mental-model ID regex, unique IDs, version validator) are already covered by upstream's schema where possible, and by the content guard for the one field that isn't.🤖 Generated with Claude Code