Skip to content

docs: add TrueNAS v1.9.6 downstream trial case study - #9

Merged
RedEyeNinja-BKK merged 2 commits into
mainfrom
docs/truenas-v196-trial-case-study
Aug 8, 2026
Merged

RedEyeNinja-BKK merged 2 commits into
mainfrom
docs/truenas-v196-trial-case-study

Conversation

@RedEyeNinja-BKK

Copy link
Copy Markdown
Owner

docs: add TrueNAS v1.9.6 downstream trial case study

Base SHA: 77c2acc8b3d926bfbf5022754e4f737499b6c21c (current post-release main)
Head SHA: e042d9b5b60011506972f47aa158372bf39b2cd0
Governing thread: Issue #8
Precedent: Issue #1 (Proxmox Manager trial, 2026-08-08)
Release baseline: Process Engine v1.9.6 (65eaba888322609afa9986ff1d19e959317f5e28)

Scope

Sanitized downstream trial evidence, one file only:
case-study/case-study-truenas-manager-v1.9.6.md

Evidence hierarchy is explicit: generation (Pattern→Review), behavioral package (T1-T10 with the intended persona + child skill pre-injected), and runtime/interoperability (primary Turnstone path + two worker-runtime executions of the same generated package).

The two worker-runtime executions are not three generation trials - they are runtime/interoperability runs of one generated package.

Contents

  • Part 1 - generation (intent, material, pipeline)
  • Part 2 - behavioral trial T1-T10 (direct results; activation/routing reported separately as UNPROVEN)
  • Part 3 - runtime/interoperability (primary + worker runtimes A/B; restricted 31-tool surface; read proof; rejected-write proof; restart/fresh-read-back)
  • Part 4 - failures and corrections preserved (tool-less first run, MCP second reload, risk_tier forensic result, Review spec-compliance gap, worker config/run-retention, manifest classification, identity placement)
  • Part 5 - lessons by layer (Process Engine / generated package / Turnstone / MCP / worker runtime / operator-supplied adapter / trial method)
  • Part 6 - what it does NOT prove

Sanitization

No private addresses, internal host names, credential paths, session/ledger identifiers, or deployment-specific values. Worker runtimes named generically (Worker runtime A / B). Detailed/private source retained locally.

Validation

python3 tools/validate.py --repo .STRUCTURAL VALIDATION PASS.

Confirmations

  • Evidence/content only - no persona, skill, reference, template, validator, workflow, or release/version change.
  • No Turnstone/runtime mutation; no TrueNAS mutation.
  • Do not merge.

Sanitized downstream evidence record for the TrueNAS Manager package trial
(2026-08-09), Process Engine v1.9.6 (release 65eaba8). References Issue #8
(governing thread) + Issue #1 (Proxmox precedent).

- Part 1 generation; Part 2 behavioral T1-T10; Part 3 runtime/
  interoperability (primary + worker runtimes A/B, restricted 31-tool
  surface, read proof, rejected-write proof, restart/fresh-read-back);
  Part 4 failures (tool-less first run, MCP second reload, risk_tier
  forensic, Review spec-compliance gap, worker config/run-retention,
  manifest classification, identity placement); Part 5 lessons by layer;
  Part 6 what it does NOT prove.

Evidence-only. No persona/skill/reference/template/validator/workflow/
version change. Sanitized: no private addresses, hostnames, credential
paths, session/ledger IDs, or deployment-specific values. STRUCTURAL
VALIDATION PASS.

@RedEyeNinja-BKK RedEyeNinja-BKK left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Independent evidence review on exact head e042d9b5b60011506972f47aa158372bf39b2cd0.

The evidence structure is strong and the PR is correctly one-file / evidence-only. Exact-head structural-validation run 31279391132 is PASS.

Before merge, please correct two evidence-level issues:

  1. Do not state that full Trial PASS exists. This document correctly establishes that activation/routing is UNPROVEN because the harness pre-injected the intended child skill and no independent trigger/near-miss routing set was recorded. Under the v1.9.6 Trial contract, that means the behavioral task suite is 10/10 PASS but the overall PE Trial contract was INCOMPLETE/UNPROVEN at Ship time. In the manifest-classification section, Review PASS, Trial PASS, explicit operator GO ... all of which exist currently contradicts the audit. Preserve the historical recorded Trial PASS label if useful, but distinguish recorded verdict from audit verdict and do not treat the former as sufficient proof.

  2. Distinguish recorded Review PASS from audit-valid Review status. The audit confirms literal non-spec SKILL.md frontmatter plus no native parse receipt, while v1.9.6 Review requires allowed fields only and native parse. So Review PASS existed as a recorded verdict, but the evidence audit shows it should have been REVISE. The case study should say that plainly instead of allowing the recorded label to imply the Review contract was satisfied.

These are evidence corrections, not new Process Engine behavior. After correction, I expect the case-study PR to be reviewable as a faithful record of the trial—including that the package behaved well and shipped, while the PE Review/Trial gates were not actually satisfied to their own v1.9.6 contract.

I would keep the product-change definition separate from this evidence PR.

Per review 4889824743 + Issue #8 steering 5228283995.

- Review: preserve recorded PASS; audit verdict = should have been REVISE
  (non-spec frontmatter, no native-parse receipt).
- Trial: distinguish behavioral 10/10 PASS / recorded overall PASS / audit
  INCOMPLETE (activation/routing unrun) - full Trial PASS not proven at
  Ship.
- Ship: authoritative proof = explicit operator GO + deployed objects +
  read-back; recorded Review/Trial PASS over-credited by audit; Ship not
  undone.

Evidence-only; structural validation PASS.

@RedEyeNinja-BKK RedEyeNinja-BKK left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Independent re-review on corrected exact head 0f73b14da902a7ad3a76879806dbd3c4851f3f94.

PASS as an evidence/documentation PR. The two prior evidence defects are corrected: (1) recorded Review PASS is now explicitly distinguished from the post-trial audit conclusion that Review should have returned REVISE because non-spec frontmatter existed and no native-parse receipt was found; (2) behavioral T1–T10 remains 10/10 PASS while recorded overall Trial PASS is explicitly distinguished from the audit verdict INCOMPLETE because activation/routing was unrun. Ship is accurately preserved as a real operator-approved deployment/read-back while the gate evidence over-credit is not hidden.

Scope remains one case-study file only; no Process Engine behavior is changed. Exact-head structural-validation workflow run 31279864913 concluded success.

Evidence PR recommendation: READY FOR EVIDENCE MERGE GATE. Keep the separate Process Engine implementation change on its own branch/PR.

@RedEyeNinja-BKK
RedEyeNinja-BKK merged commit d0e0a2d into main Aug 8, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant