Attribute attestation signatures, and prove the evidence layer fires - #106
Draft
b-macker wants to merge 2 commits into
Draft
Attribute attestation signatures, and prove the evidence layer fires#106b-macker wants to merge 2 commits into
b-macker wants to merge 2 commits into
Conversation
living-script_extended demonstrated 23 governance levels and produced zero signed attestations: it had telemetry.tamper_evidence on and no audit section at all. It also built a tamper-evident chain that nothing ever verified, and archived telemetry and transcript but no audit file — two more instances of the "configured but unobserved" gap its own header records as the reason levels 19b/21/22/23 exist. Validating the config against the stub before writing any of it surfaced a real defect. ed25519Fingerprint() reads with PEM_read_bio_PUBKEY and refuses private keys by design, but all three emit*Attestation sites handed it the signing key, so fingerprinting silently returned "" into the surrounding catch. Every signed attestation NAAb has written carries a signature that cannot be attributed to a key. Fixed by fingerprinting the derived public half via the existing ed25519PublicFromPrivate(), matching how governance_engine.cpp:4123 already does it; the fingerprint now equals the one the trust store reports for the same key. The C-05 assertion merged in #104 could not have caught this: it greps '"key_fingerprint":"' which also matches an empty value. It now requires hex content. The new level asserts fingerprint content for the same reason. Two attestation kinds go to different files behind different gates. Refusal attestations land in telemetry; execution attestations go through logAuditEvent to the audit file, gated additionally on audit.level != "none" which defaults to none. That distinction decides the level's design: refusal attestations only exist when governance blocked something, so asserting a floor on them would demand the agent misbehave. Execution attestations fire once per send, so they are the countable population. Refusals are therefore checked for AGREEMENT with their attestations — zero of both passes. Counting stays in grep/wc because decision_snapshots pushes telemetry past a megabyte and it must not enter script memory. The audit patterns need -F: process.run passes argv with no shell, so grep sees one backslash and its BRE reads \" as a plain ", which never matches the escaped attestation body. Verified without live keys: 3 sends produce 3 attestations, all signed with a non-empty fingerprint, snapshots on every SEMANTIC_TURN and OUTPUT_ADMISSIBILITY_EVAL, both chains verifying. Each L24 assertion was checked against its own degraded case and fails only there. Full suite: 441 tests, 0 unexpected failures. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ELUfjXZvx8kzXo1UJjrAhC
NAAb Governance Report
All governance checks passed! Generated by NAAb Governance Engine v4.0 |
…count
The live keyed run reported L24-02 failing: 22 execution attestations against
21 script-counted sends. The denominator was wrong, but adding commits to it
would not have fixed the check — 21 + 2 commits is 23, not 22.
Exact equality is not a sound invariant here, for three independent reasons:
- agent.commit() attests as well as agent.send() (agent_impl.cpp:5382), so
total_sends was never the right denominator.
- safe_send() catches a throw but the script still increments total_sends, so
a send blocked before reaching emitAttestation counts as a send and
produces no attestation. That makes equality swing on agent behaviour.
- agent.run/batch/fan_out/pipeline each bump the counter by one or two, and
whether each attests is a separate question per API.
The second reason is exactly why L24-05 asserts agreement rather than a floor
for refusal attestations. I reasoned it out for that check and then wrote the
neighbouring one as a hard equality anyway.
L24-02 now asserts what holds regardless of behaviour: attestations exist,
every one carries a known action (send + commit must sum to the total, so an
unaccounted action type fails), and attested sends cannot outnumber attempted
sends (a phantom attestation fails). The script reports the send/commit split
and the script-side count for forensics instead of asserting on them.
Verified against the live-reported numbers: 22 = 20 send + 2 commit with 21
attempted now passes, while record_attestations off, an unaccounted action
type, and phantom attestations each still fail.
The rest of the live run confirmed the change: 22 signed with 0 empty
fingerprints (the fix in this PR), 43 snapshots = 21 semantic + 22 OA, both
chains verified, the audit section survived 15 accepted operator rewrites with
no ratchet rejection touching provenance, and telemetry grew 550KB -> 1.3MB.
Full suite: 441 tests, 0 unexpected failures.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELUfjXZvx8kzXo1UJjrAhC
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
living-script_extendeddemonstrated 23 governance levels and produced zero signed attestations —telemetry.tamper_evidencewas on with noauditsection at all. It also built a tamper-evident chain nothing ever verified, and archived telemetry and transcript but no audit file. Two more instances of the "configured but unobserved" gap its ownrun.shheader records as the reason levels 19b/21/22/23 exist.The defect this surfaced
Validating the config against the stub before writing the level found a real bug:
ed25519Fingerprint()reads withPEM_read_bio_PUBKEYand refuses private keys by design, but all threeemit*Attestationsites handed it the signing key. Fingerprinting silently returned""into the surroundingcatch (...) {}. Every signed attestation NAAb has ever written carries a signature that cannot be attributed to a key.Fixed by fingerprinting the derived public half via the existing
ed25519PublicFromPrivate(), matching howgovernance_engine.cpp:4123already does it. The fingerprint now equals what the trust store reports for the same key:The C-05 assertion merged in #104 could not have caught this — it greps
'"key_fingerprint":"', which also matches"". Tightened to require hex content. The new level asserts fingerprint content for the same reason.Why the level is shaped this way
The two attestation kinds go to different files behind different gates:
provenance.enabled+record_attestationsaudit.level != "none"(defaults tonone)Refusal attestations only exist when governance actually blocked something, so asserting a floor on them would demand the agent misbehave. Execution attestations fire once per
agent.send(), so they're the countable population. Refusals are checked for agreement with their attestations instead — zero of both is a pass.That's also why
audit.leveland archiving the audit file are required, not optional: without them the level has nothing deterministic to assert.Changes
src/runtime/governance_reports.cpp— fingerprint the derived public key at all 3 attestation sites.trust_store.cppuntouched; it already receives a public PEM.tests/governance_v4/test_nonformation_proof.sh— C-05 requires[0-9a-f]{16,}, not mere presence.src/govern.json—auditsection (level: basic,tamper_evidence,provenancewithsigning_key_env, not a literal path — the key lives in an ephemeral$TEST_TMP) plustelemetry.decision_snapshots.src/living-script.naab— newEVIDENCE_AUDITphase. Counting stays in grep/wc becausedecision_snapshotspushes telemetry past a megabyte and it must not enter script memory. Added a-Fhelper:process.runpasses argv with no shell, so grep sees one backslash and its BRE reads\"as a plain", which never matches the escaped attestation body.run.sh— Level 24 assertions, archiveaudit.jsonl, and verify both chains.Test Plan
bash run-all-tests.sh— 441 tests, 0 unexpected failurestest_nonformation_proof.sh15/15 with the tightened C-05tests/security/test_error_msg_leaks.sh— 874/0 (attestation code touched)End-to-end without live keys, via
tests/helpers/agent_stub.py: 3 sends → 3 execution attestations, all signed with non-empty fingerprints,cdd_snapshoton everySEMANTIC_TURNandOUTPUT_ADMISSIBILITY_EVAL(6 = 3+3), both chainsChain verified:The real phase code, extracted verbatim from the script, run against those real files — correct markers
Every L24 assertion checked against its own degraded case; each fails only there:
record_attestationsoffdecision_snapshotsoffparseclean;checkdiagnostics unchanged at 35;run.shpassesbash -n;govern.jsonvalid JSONLive keyed run — not possible here. Two things to confirm on the first real run: that the
auditsection survives the operator'scat→json.parse→json.stringify→ re-sign round-trip (if dropped,record_attestationsgoes true→false and the evidence ratchet rejects every later reload), and that noCONFIG_ADJUSTMENTcarriesaccepted:falsewithreason:ratchet.Notes
L24-03 reads PASS when there are zero attestations (0-of-0 signed), but L24-02 catches that case, so the pair is complete.
Left deliberately out of scope: the pre-existing
NAAB_SIGN_PATH/NAAB_SIGNING_KEYnaming mismatch; andtaint_tracking/temporal_coupling/integrity, each of which needs a deliberate observable sink to avoid being inert.Generated by Claude Code