docs: add TrueNAS v1.9.6 downstream trial case study - #9
Conversation
Sanitized downstream evidence record for the TrueNAS Manager package trial (2026-08-09), Process Engine v1.9.6 (release 65eaba8). References Issue #8 (governing thread) + Issue #1 (Proxmox precedent). - Part 1 generation; Part 2 behavioral T1-T10; Part 3 runtime/ interoperability (primary + worker runtimes A/B, restricted 31-tool surface, read proof, rejected-write proof, restart/fresh-read-back); Part 4 failures (tool-less first run, MCP second reload, risk_tier forensic, Review spec-compliance gap, worker config/run-retention, manifest classification, identity placement); Part 5 lessons by layer; Part 6 what it does NOT prove. Evidence-only. No persona/skill/reference/template/validator/workflow/ version change. Sanitized: no private addresses, hostnames, credential paths, session/ledger IDs, or deployment-specific values. STRUCTURAL VALIDATION PASS.
RedEyeNinja-BKK
left a comment
There was a problem hiding this comment.
Independent evidence review on exact head e042d9b5b60011506972f47aa158372bf39b2cd0.
The evidence structure is strong and the PR is correctly one-file / evidence-only. Exact-head structural-validation run 31279391132 is PASS.
Before merge, please correct two evidence-level issues:
-
Do not state that full
Trial PASSexists. This document correctly establishes that activation/routing is UNPROVEN because the harness pre-injected the intended child skill and no independent trigger/near-miss routing set was recorded. Under the v1.9.6 Trial contract, that means the behavioral task suite is 10/10 PASS but the overall PE Trial contract was INCOMPLETE/UNPROVEN at Ship time. In the manifest-classification section,Review PASS, Trial PASS, explicit operator GO ... all of which existcurrently contradicts the audit. Preserve the historical recorded Trial PASS label if useful, but distinguish recorded verdict from audit verdict and do not treat the former as sufficient proof. -
Distinguish recorded Review PASS from audit-valid Review status. The audit confirms literal non-spec SKILL.md frontmatter plus no native parse receipt, while v1.9.6 Review requires allowed fields only and native parse. So
Review PASSexisted as a recorded verdict, but the evidence audit shows it should have beenREVISE. The case study should say that plainly instead of allowing the recorded label to imply the Review contract was satisfied.
These are evidence corrections, not new Process Engine behavior. After correction, I expect the case-study PR to be reviewable as a faithful record of the trial—including that the package behaved well and shipped, while the PE Review/Trial gates were not actually satisfied to their own v1.9.6 contract.
I would keep the product-change definition separate from this evidence PR.
Per review 4889824743 + Issue #8 steering 5228283995. - Review: preserve recorded PASS; audit verdict = should have been REVISE (non-spec frontmatter, no native-parse receipt). - Trial: distinguish behavioral 10/10 PASS / recorded overall PASS / audit INCOMPLETE (activation/routing unrun) - full Trial PASS not proven at Ship. - Ship: authoritative proof = explicit operator GO + deployed objects + read-back; recorded Review/Trial PASS over-credited by audit; Ship not undone. Evidence-only; structural validation PASS.
RedEyeNinja-BKK
left a comment
There was a problem hiding this comment.
Independent re-review on corrected exact head 0f73b14da902a7ad3a76879806dbd3c4851f3f94.
PASS as an evidence/documentation PR. The two prior evidence defects are corrected: (1) recorded Review PASS is now explicitly distinguished from the post-trial audit conclusion that Review should have returned REVISE because non-spec frontmatter existed and no native-parse receipt was found; (2) behavioral T1–T10 remains 10/10 PASS while recorded overall Trial PASS is explicitly distinguished from the audit verdict INCOMPLETE because activation/routing was unrun. Ship is accurately preserved as a real operator-approved deployment/read-back while the gate evidence over-credit is not hidden.
Scope remains one case-study file only; no Process Engine behavior is changed. Exact-head structural-validation workflow run 31279864913 concluded success.
Evidence PR recommendation: READY FOR EVIDENCE MERGE GATE. Keep the separate Process Engine implementation change on its own branch/PR.
docs: add TrueNAS v1.9.6 downstream trial case study
Base SHA:
77c2acc8b3d926bfbf5022754e4f737499b6c21c(current post-release main)Head SHA:
e042d9b5b60011506972f47aa158372bf39b2cd0Governing thread: Issue #8
Precedent: Issue #1 (Proxmox Manager trial, 2026-08-08)
Release baseline: Process Engine v1.9.6 (
65eaba888322609afa9986ff1d19e959317f5e28)Scope
Sanitized downstream trial evidence, one file only:
case-study/case-study-truenas-manager-v1.9.6.mdEvidence hierarchy is explicit: generation (Pattern→Review), behavioral package (T1-T10 with the intended persona + child skill pre-injected), and runtime/interoperability (primary Turnstone path + two worker-runtime executions of the same generated package).
The two worker-runtime executions are not three generation trials - they are runtime/interoperability runs of one generated package.
Contents
risk_tierforensic result, Review spec-compliance gap, worker config/run-retention, manifest classification, identity placement)Sanitization
No private addresses, internal host names, credential paths, session/ledger identifiers, or deployment-specific values. Worker runtimes named generically (Worker runtime A / B). Detailed/private source retained locally.
Validation
python3 tools/validate.py --repo .→ STRUCTURAL VALIDATION PASS.Confirmations