[bryantgillespie]

Brings the skills to Directus 12.6.0, adds the directus-self-hosting and directus-content skills, and adds behavior evals that show where each skill actually changes what an agent builds. Skills that models handle fine on their own were cut back.

Every claim is verified against the 12.6.0 source or a live 12.6.0 instance (SQLite, plus Postgres where behavior differs).

Skills

• New directus-self-hosting: Docker Compose, configuration, file storage and email, scaling past one instance, backups, Directus Cloud.
• New directus-content: editing content the way an editor does, over MCP or the API. Covers staging a change in a content version (MCP and REST item writes always go live), relational edits with row ids, files, and deletes.
• Version notes per major (directus/references/versions-v9 to v12), so agents on older instances read what changed first.
• New references: indexes, schema changes on live data, starting from a starter, existing SQL databases, multi-tenancy, query performance, member sign-up, import and export, audit and restore, AI settings, JSON field filters.
• Descriptions lead with "Load first, before exploring code", which took Haiku from 22/37 to 34/37 trigger cases.
• "House rule" markers are now "Recommended", and tables are compact (the validator enforces it).

Cut back

Behavior evals showed models pass these tasks without the skill, so they were trimmed:

• directus-sdk and directus-environment-sync are down to their traps and commands, from about 1,690 lines combined to 192. Both still pass all their cases with the plugin.
• Index, flow recipe, loop, and content-editing references lost the generic how-to.

Traps found while testing

• A Run Script linked directly to a condition's reject path fails with illegal access (500).
• A Run Script throw reaches the caller as a 500; throw-error gives a 400 with your message.
• On a create, the item rule is ignored and validation only sees the payload. So a per-org level (admin in one org, viewer in another) needs a filter Flow.
• A UUID primary key without meta.special: ["uuid"] gets no generated id. Creates fail with 400 Value is required.
• Re-sending a translations array without ids creates new rows and orphans the old ones.
• A content version that saves a page-builder edit without the block's collection breaks every read of that version, and versioned reads must select blocks.collection.
• Existing databases: tables come in with meta: null and are configured with PATCH. Tables without a single-column key can't be used. Enums come in as unknown and a bad value leaks SQL; field validation fixes that on the core tier.
• _json filters compare query-string numbers as text. json() doesn't work in filter or aggregate.
• Extension auto reload misses the first build. Docker Desktop bind mounts need polling.
• The MCP trigger-flow prompt documents parameters the tool rejects (flowDefinition, flowId) and webhook flows it can't run. The tool takes id and runs only active manual flows.
• The SDK's aggregate() accepts a top-level filter without a type error and ignores it.

Behavior evals

behavior-evals/ starts a fresh Directus container per run, seeds it, gives headless Claude a task with and without the plugin, then checks the instance and files over the API. 36 cases cover modeling, access, flows, Insights, SDK, sync, self-hosting, upgrades, extensions, visual editing, existing databases, content editing over REST and MCP, and multi-tenant access. Licensed cases confirm the license is active before the agent starts.

• A run passes when every functional check passes. Our conventions (naming, key types, on_delete, clean 4xx refusals) are scored separately as recommended checks.
• Runs where the agent never started (usage limit, API error) count as errors, not failures.
• Release read: node behavior-evals/run.mjs --clean --model haiku,sonnet --runs 3.

Where the plugin changes the result (runs passing out of 3, with plugin / without):

Case	Haiku	Sonnet
upgrade-project	3 / 0	3 / 0
selfhost-compose, selfhost-scale	3 / 0	3 / 0
visual-editing-preview	3 / 0	3 / 0
insights-author-dashboard	3 / 0	3 / 0
flows-publish-guard	3 / 0	3 / 0
page-builder	3 / 1	3 / 0
json-filters	3 / 1	3 / 2
translations	3 / 0	3 / 3
internal-tools-approvals	2 / 0	3 / 3
content-draft	3 / 0	3 / 3
existing-database-hard	3 / 1	3 / 3
extensions-endpoint	3 / 1	3 / 3
access-tenant-roles	3 / 2	3 / 3

Ties on both models (SDK, sync, simple flows, content edits, basic access) are kept as regression guards.

Checks

• node scripts/validate.mjs --strict: 0 errors, 0 warnings.
• Trigger evals on Haiku, 3 runs each: 32 of 39 cases pass every run. The other 7 pass 1 or 2 of 3; in the misses Haiku called no skill at all. Every SDK and sync case passes.

Follow-ups

• Consolidate overlap the review flagged: one owner for version notes, framework and content-version duplication, traps repeated across skills, reference-to-reference links.
• Run the evals on Opus too.