External, version-pinned plugin for SPECA.
It implements the lean property-generation method behind the provider seam
landed in speca#87: it turns the
formally-verified Casper FFG theorems in
NyxFoundation/gasper-lean4
into SPECA 01e security properties.
Status: M-lean-provenance workstreams A + B + D (issues #3, #4, #6; epic #2), on top of the M0-M3 Core-retargeted baseline. Per the gasper-lean4 maintainer, ~70-80% of the proved substance lives in
GasperBeaconChain.Core.*(Theories + Lemmas), not the thin, still-growingExecutableapplication layer, so the target set is Core-centric (18 Core theorems + 7 Executable decidable-checker counterparts = 25 theorems). The Lean->plugin boundary now carries the real proof content (statement, hypothesis telescope, conclusion, referenced constants, gasper-local axioms, proof provenance, verbatim proof source), and each theorem lowers to one01eproperty per must-establish precondition. First live CI export (2026-07-18): 53 must-establish hypotheses across the 25 theorems -> 61 properties, verbatim proof source extracted for all 25, zero gasper-local axioms, zero type-consistency mismatches. CI imports bothCore.AllandExecutable.All, runs the exporter end-to-end, and certifies every targetlean_status: proved(sorry-free, choice-free, native-free) from realcollectAxiomsoutput.Stage ② checklist overlay (speca#88 stage 2): on top of the mechanical lowering,
theorem_map.jsoncarries 15CHK-*entries — LLM-synthesized implementation invariants (first draft, under review) that descend from the same proved theorems but target the arithmetic / type-width / bounds / termination / resource failure surface documented by real client bugs inethereum-vuln-dataset(cited per entry). They lowerverbatim(never decomposed per must-establish hypothesis) into their ownchecklist-high-angleshard, and emitlean_status: descends-from-proved— never plainproved: the hand-written text is not Lean-verified, only the theorem it descends from is. Design:docs/high-angle-checklist.md.
| track | source | layer | map | workspace |
|---|---|---|---|---|
| gasper | gasper-lean4 (git dep) |
consensus — Casper FFG accountable safety | theorem_map.json |
lean/ (Lean 4.31.0) |
| ethtotal | eth-total-supply-safety (submodule at external/) |
execution — total-supply & ledger accounting | theorem_map_ethtotal.json |
lean-ethtotal/ (Lean 4.33.0-rc1) |
Everything below describes the gasper track unless stated otherwise; the second
track has its own end-to-end write-up in
docs/ethtotal-track.md. Short version: all 3333
theorems of the EthTotal development are exported and certified
(lean_status: proved, zero non-builtin axioms), every one of them is bucketed
by a reviewable triage so the Lemmata/ layer is swept rather than sampled, 176
become base entries (55 reviewed + 121 derived), and the CRITICAL/HIGH ones are
concretized into a CHK-* implementation checklist that the stage-2 judge →
improve loop sharpens against an execution-layer slice of
ethereum-vuln-dataset.
Spec anchoring is per-layer: the consensus labels resolve through
data/anchor_map.json to consensus-specs pyspec symbols, the execution-layer
ones through data/anchor_map_execution.json to
ethereum/execution-specs
symbols, each row verified (file + line) against a pinned checkout by
tools/build-el-anchor-map.py.
The two tracks share one exporter: SpecaExport is parameterized by a
ProjectConfig (namespace, project name, model assumptions), with
gasperConfig holding exactly the previous hardcoded values — the gasper export
is byte-compatible, gasper_axioms included.
Per speca#87/#88 the Lean/lake toolchain stays out of core speca. speca
references this repo as an officially-certified, version-pinned plugin; its
lean provider invokes the CLI below across the seam.
gasper-lean4 (proved theorems)
│
▼
lean/ ── lake exe speca-export ──► theorem-health JSON (Stage A+B: Lean-only)
│ collectAxioms per target theorem → lean_status: proved | unknown
▼
src/speca_lean4/ ── speca-lean4 emit-01e ──► 01e_PARTIAL JSON (Stage C: Python)
│ theorem_map.json (§3 mapping) + BUG_BOUNTY_SCOPE.json + 01b subgraphs
▼
speca (02c → 03 → 04 audit) ──► #92 Kurtosis reproduction
- Lean side (
lean/) does only the part that needs Lean: resolve each target theorem, collect the axioms its proof depends on (the same mechanism as gasper-lean4's#mr_audit_json), classify itproved(nosorry) orunknown, and extract the proof content (statement, hypothesis telescope with depend-allowed/must-establish tags, conclusion, referenced constants, proof term + verbatim source). Nothing about01elives here. - Python driver (
src/speca_lean4/) owns the lowering semantics (B1-B5 below), scope resolution, andcoversmatching.theorem_map.jsonis the tuning overlay — severity calibration, covers hints, scope, labels, shards — editable without recompiling Lean; the statement/hypothesis content itself comes from the Lean export, not from the map.
speca-lean4 emit-01e \
--scope outputs/BUG_BOUNTY_SCOPE.json \
--map theorem_map.json \ # optional; defaults to repo root
--subgraphs 'outputs/01b_PARTIAL_*.json' \ # optional; for covers resolution
--gasper-ref <git-sha> \ # optional; pins gasper-lean4
( --health-json health.json | --run-lean ) \
--out outputs/01e_PARTIAL_lean.jsonProof-health source (Stage B) is one of:
--health-json— a precomputedlake exe speca-exportoutput (CI produces it).--run-lean— runlake exe speca-exportnow (needs the Lean toolchain).- neither — every property is emitted
lean_status=unknown(with a warning); useful for a dry mapping check without Lean.
For implementation-facing CL obligations and cross-layer EL obligations, use the causal projection path:
speca-lean4 emit-projected-01e \
--scope outputs/BUG_BOUNTY_SCOPE.json \
--health-json health.json \
--target-layer both \
--out outputs/01e_PARTIAL_gasper_projected.json
speca-lean4 refine-projected-01e \
--input outputs/01e_PARTIAL_gasper_projected.json \
--out outputs/01e_PARTIAL_gasper_projected_refined.jsonIts reviewed data/projection_map.json makes the
theorem -> owned input -> boundary obligation -> code surface chain explicit.
It fails on unclassified theorems, records reviewed EL not-applicable
decisions, and keeps EL properties at
descends-from-proved-via-unproved-bridge until the composition theorem is
proved. The second command recursively applies reviewed atomic split rules
while preserving that provenance and proof status. See
docs/causal-projection.md.
Output is exactly the 01e property schema. Lean-specific data is additive
only, never mutating a core field — per speca#88's contract: lean_status,
lean_artifact, kurtosis_test, label, lean_statement,
lean_hypotheses, lean_must_establish, lean_referenced_defs,
lean_referenced_defs_expanded, lean_axioms, lean_proof_provenance,
lean_proof_code, lean_precondition, lean_conclusion,
lean_type_consistency, lean_proof_source, lean_doc_string,
spec_reference.
The end-to-end execution flow and every artifact location are consolidated in
docs/pipeline.md.
label follows the ethereum-vuln-dataset controlled vocabulary
(consensus-specs section names, docs/label_design.md), and spec_reference
- the
coversfallback derive mechanically from it (C5): each label maps to its consensus-specs doc anchor and primary pyspecprocess_*symbol — no prose judgment.
Spec/code anchoring (issue #5, workstream C) is table-driven:
data/anchor_map.json is the def -> spec-symbol ->
client-code-symbol alignment table (C3) that spec_reference/covers are
derived from (C4, via src/speca_lean4/anchors.py). The client-code column is
best-effort and honest: rows are verified-<date> (confirmed by code search)
or explicitly todo, never fabricated. Declaration-site annotations
(@[speca_spec], C1/C2) are blocked on gasper-lean4 maintainer coordination
(issue #9 G2); the proposed convention is documented in
docs/spec-annotation.md. Hygiene (issue #10): the
honesty invariants are pinned by explicit tests (tests/test_honesty.py, H1),
and the local-Lean-toolchain question is evaluated in
docs/lean-toolchain.md (H2 — staying CI-only until
lean/ churn justifies local elan/lake).
lake exe speca-export emits one record per target theorem. Beyond the proof
health flags (lean_status, sorry_free, choice_free, native_free), each
record carries:
| field | source | item |
|---|---|---|
statement |
Meta.ppExpr of ConstantInfo.type |
A1 |
hypotheses |
Meta.forallTelescope; per binder: name, type (pp), head (head constant), class |
A2 |
conclusion |
pp of the telescope body — the Q the theorem guarantees | A2/B2 |
referenced_constants |
Expr.getUsedConstants on the type, filtered to GasperBeaconChain.* |
A3 |
gasper_axioms |
collectAxioms minus Lean builtins (sorryAx, Classical.choice, propext, Quot.sound, trustCompiler, native compute) |
A4 |
proof_provenance |
"automated" if the proof term references decision-procedure markers (Decidable.decide, ofReduceBool/Nat, of_decide_eq_true, Aesop.*, Omega.*), else "hand-written"; "unknown" if no value |
A5 |
proof_code |
pp of ConstantInfo.value? (pp.proofs true) |
A7 fallback |
proof_constants |
gasper-local getUsedConstants of the proof term — the proof-DAG edges |
B3 feed |
proof_source |
verbatim declaration source (term/tactic code + comments), sliced from the lake package checkout via findDeclarationRanges?; since A7+ the slice is widened upward over the contiguous leading comment block (docstring + adjacent -- comments), so the proof and its documentation travel as one pair; "" if unavailable |
A7 / A7+ (#17) |
referenced_defs_expanded |
recursively pretty-printed definitions of the gasper-local constants the statement references — structure fields (projection signatures), inductive constructors, def signature+body — as [{name, kind, pp}]. Honestly bounded, with the caps stated in code (lean/SpecaExport/Basic.lean): breadth-first to depth 2, at most 24 definitions per theorem, 4000-char pp cap with an explicit truncation marker, deduped; mathlib/Lean-core constants and compiler-generated auxiliaries are never expanded |
A3+ (#16) |
doc_string |
the declaration's docstring via findDocString?; "" when the theorem has none — empty, never fabricated |
A7+ (#17) |
Documented in lean/SpecaExport/Basic.lean; provisional and expected to be
tuned with the gasper maintainers:
- instance-implicit binders -> depend-allowed (typeclass plumbing);
- non-
Propbinders -> depend-allowed (model parameters:τ,stake,vset,parent,genesis,st, ...); Prophypotheses whose head predicate is a fixed world/model assumption (two_thirds_good,good_votes,blocks_exist_*,target_height_bound) -> depend-allowed;- every other
Prophypothesis -> must-establish: a computed/structural fact (k_finalized ...,justified ...,quorum_2 ..., height inequalities, ...) the implementation must preserve for the theorem's guarantee to transfer.
- B1 — decomposition. A theorem lowers to N properties, one per
must-establish hypothesis (
<base-id>-me<i>), NOT one per theorem. Depend-allowed hypotheses are context, never invariants. Theorems with no must-establish hypothesis (the Iff-shaped decidable checkers) lower 1:1. Per the MTG principle the must-establish set grows with implementation sophistication — 54 properties from 25 theorems on the sample fixture. - B2 — neutral audit result. Each assertion reads:
implementation must preserve [P]; if so, <theorem> guarantees [Q]with P and Q pretty-printed from Lean — an investigation result, not a good/bad verdict. - B3 — proof-DAG severity. Top-level conclusions keep the theorem_map
severity; a lemma inherits the maximum severity of the target theorems whose
proofs (transitively) depend on it, via
proof_constants. Upward-only — nothing is downgraded, and map severities were not relabeled. - B5 — type-consistency gate. Each lowered precondition's head constant
must be among the theorem's referenced constants; verdict
ok/mismatch/uncheckedinlean_type_consistency. Mismatches are warned at emit time and fail the CI end-to-end step.
Honesty invariants are unchanged: sorry -> unknown, unresolved target ->
unknown + CI failure, severities never tuned against the benchmark
distribution.
The Executable layer is the reproduction bridge to speca#92
(kurtosis-harness): its
decidable Bool checkers (slashedB, justifiedB, notSlashedB,
goodQuorumAtB, qIntersectionWitnessB) and constructive witnesses
(accountable_safety_witnessB, k_accountable_safety_witnessB,
plausible_liveness_construct_extension) turn an existence-shaped Core proof
into something a devnet can evaluate.
speca-lean4 emit-kurtosis \
--scope tests/fixtures/bug_bounty_scope.sample.json \
--health-json lean/health.json \
--fixtures-dir outputs/kurtosis \ # <label>/<property_id>/{devnet,assertion}.scaffold.json
--out outputs/01e_kurtosis.json # 01e with checker/witness/kurtosis_test populated- E1
data/checker_map.jsonlinks each theorem to its Executable checker(s), the proved..._iffcorrectness theorems, and its witness (when one exists); the emitted property surfaceschecker/witness. - E3 one
kurtosis_testfixture SCAFFOLD per checker-linked property (devnet placeholder + assertion stub referencing the checker). Fixtures are clearly scaffolds ("scaffold": true, verdict null) — not runnable. - E6
data/evidence_seeds.jsonattaches label-matched ethereum-vuln-dataset findings (pre_fix_code/files_changedexcerpts) as implementation-linked evidence seeding the reproduction target. - E2 / E5 (devnet bring-up +
KurtosisVerificationBackendhandoff) are blocked on speca#92 and designed — not built — indocs/kurtosis-bridge.md.
Honesty: kurtosis_test / checker / witness are non-null only where a
real Executable checker exists; the pure-arithmetic bound and definitional Core
theorems are honestly null.
# restore the granularity reference (426-file benchmark release)
gh release download bench-rq2a-20260508-speca --repo NyxFoundation/speca
tar --zstd -xf bench-rq2a-20260508-speca.tar.zst # -> speca/01e_*.json (16 files)
speca-lean4 emit-01e --scope scope.json --health-json health.json \
--out outputs/01e_PARTIAL_lean.json \
--out-dir outputs/01e_lean/ # sharded: 01e_PARTIAL_<shard>.json
speca-lean4 verify-precision \
--ours outputs/01e_PARTIAL_lean.json \
--ours-dir outputs/01e_lean/ \ # per-shard props/file granularity
--benchmark-dir speca \
--out precision_report.jsonMeasures granularity vs the benchmark corpus (props/file and assertion-length
z-scores, severity KL divergence, vocabulary conformance) and label-grounded
recall vs the dataset's structured label vocabulary (workstream D, issue
#6). The prose judgment table data/findings_map.json is deprecated (its
strict/lenient numbers are still reported as recall_prose_deprecated for
continuity); recall is now computed from three reviewable data files:
data/ethereum_vulns.csv— vendored consensus-domain slice ofethereum-vuln-dataset(revision pinned indata/ethereum_vulns.meta.json): all 37 rows whoselabelis one of the three consensus-specs areas the FFG target set maps to. Vendored unfiltered beyond the label so the denominator derivation stays reproducible.data/label_match_rules.json— (a) the STRUCTURAL in-domain filter (D1,v1-narrow, grown gradually): label + root_cause + attack_path + severity, no per-finding prose. Current denominator: 9 in-domain findings of the 37 slice rows. (b) coverage rules per(label, root_cause)cell, each naming the base property ids that catch the class and a rationale; unlisted cells count uncovered.data/recall_gaps.json— the D2 gap loop: every uncovered in-domain finding triagednew_target(which precondition/theorem WOULD catch it — a concrete growth target) orout_of_model(honestly outside the FFG remit).
# label recall alone needs no benchmark corpus — CI runs it against the real
# emitted 01e right after emit-01e:
speca-lean4 verify-recall --ours outputs/01e_PARTIAL_lean.json --strictHonesty guards, all evaluated against the REAL emitted 01e: a rule's
covered_by id must actually be emitted (base id or a B1 -me<i>
refinement) with the same label, else the finding counts uncovered;
--strict fails CI on untriaged uncovered findings, stale gap entries, or
rules naming non-emitted properties. Current numbers (domain v1-narrow):
label recall 0.556 (5 of 9 covered; gaps: 2 new_target — the
exact-arithmetic / bounded-representation preconditions the slashing-overflow
class needs — and 2 honestly out_of_model: slashing-protection DB and REST
API surfaces). The deprecated prose table's strict recall was 0.333 on a
denominator of 3; the numbers are not directly comparable — the label-grounded
denominator is structural, larger, and grows deliberately.
Post-decomposition, from the live CI export (2026-07-18, 61 properties): the
six protocol-area shards carry 9 / 10 / 13 / 10 / 8 / 11 properties
(safety-accountable, safety-cases, safety-bound-witness,
finality-justification, finality-quorum, finality-liveness) — all within the
benchmark 1-sigma props/file band (11.62 +/- 3.72). The lean job uploads
health.json + the emitted 01e as a CI artifact; shards are re-tunable in
theorem_map.json without touching Lean or Python. Real severity
distribution after B3 DAG derivation: CRITICAL 12 / HIGH 30 / MEDIUM 19.
Known cosmetic limitation: conclusions/hypotheses pretty-print with default
exporter options, so some arithmetic renders applied-instance style
(instLENat.le ...) rather than infix notation; faithful, just verbose. The
historical M0-M3 table below predates the B1 decomposition (25 x 1:1
properties):
| metric | Core sharded (M3) | Core single-file | prior (7 props) | benchmark |
|---|---|---|---|---|
| schema validity | 100% | 100% | 100% | — |
| vocabulary conformance | 100% | 100% | 100% | — |
| properties per file | 12.5 ± 0.71 (safety 12 z=+0.10, finality 13 z=+0.37; both in 1-sigma) | 25 (z = +3.6) | 7 (z = −1.24) | 11.62 ± 3.72 |
| assertion length | mean z = −0.2 | mean z = −0.2 | mean z = −0.49 | 93.55 ± 14.62 chars |
| severity KL(ours‖bench) | 0.6004 nats | 0.6004 nats | 0.3015 nats | CRITICAL/HIGH/MEDIUM = 95/81/10 |
| recall (strict / lenient) | 0.333 / 0.667 | 0.333 / 0.667 | 0.0 / 0.667 | 3 in-domain of 14 consensus-layer findings |
Honest reading. The Core retarget captures the substantive Core theorems and
raised strict recall 0.0 → 0.333: the Core SlashableBound trio pins exact
validator-set weight arithmetic, giving full coverage of the Electra
effective-balance finding (GHSA-wm9c-xvqq-5c28) that the Executable-only set
could only cover partially. M3 then fixed granularity by structure, not
relabeling — emit-01e --out-dir shards the 25 properties into two coherent
protocol-area files (safety 12, finality 13), each landing within the
benchmark's 1-sigma props/file band (single-file emit is kept for speca's
provider call). The severity KL is deliberately unchanged: the
justification/quorum lemmas are honestly MEDIUM and were not relabeled to game
the metric — a formal-methods-derived set simply has a different, flatter
severity profile than the CRITICAL/HIGH-heavy benchmark, which we report rather
than fake. Most consensus-layer findings (OOM/DoS, LMD-GHOST, BLS internals,
eth1 ops) remain out of the FFG formal remit by construction.
eval here is NOT recall (the #88 direction's 重要な訂正): the checklist is not
scored on reproducing dataset bugs, but on reaching the QUALITY LEVEL of a
professional audit checklist. speca-lean4 judge scores every item on five
fixed axes (specificity, implementation_readiness, generality, actionability,
granularity; blind rubric in src/speca_lean4/judge.py) and compares the
score DISTRIBUTION against the same-rubric scores of the vendored solodit
reference (data/solodit_checklist.csv, 52
professional audit items, provenance pinned in
data/solodit_checklist.meta.json).
speca-lean4 improve then loops judge -> sharpen low scorers (with the
matching data/ethereum_vulns.csv rows as teaching material — improve input,
never an eval denominator) -> re-judge, converging only when the reference
bar is met AND the last 3 rounds plateau; an unconverged run ends honestly
with converged: false. recall.py's 0.556 stays a side reference number,
outside the judge verdict. Full spec: docs/judge-loop.md.
# LLM access is injected — no API key in this repo. --llm-cmd reads the
# prompt on stdin and writes the response on stdout, e.g. an authenticated
# Claude CLI (the real run lives in .github/workflows/judge-dispatch.yml,
# self-hosted, dispatch-only; default CI wires tests/fixtures/mock_llm.py):
speca-lean4 judge --ours outputs/01e_PARTIAL_checklist-high-angle.json \
--llm-cmd "claude -p" --out judge_report.json
speca-lean4 improve --ours outputs/01e_PARTIAL_checklist-high-angle.json \
--llm-cmd "claude -p" --ref-report judge_report.json \
--out-dir improve_run --max-rounds 6
# -> improve_run/score_log.json (per-round progression, convergence verdict)
# improve_run/improved_01e.json (proposal; theorem_map.json stays canonical)cd lean
lake exe speca-export --targets targets.txt # newline-delimited theorem names
# → proof-health JSON on stdout# Python driver + mapping (no Lean needed)
pip install -e '.[dev]'
pytest -q
# Lean exporter (needs elan/lake; toolchain pinned in lean/lean-toolchain)
cd lean && lake buildThe substantive results — the primary targets — are the Core theorems
(GasperBeaconChain.Core.*, in Theories/ and Lemmas/). Note the whole Core
layer shares the flat GasperBeaconChain.Core namespace; Theories/ and
Lemmas/ are file paths, not namespace segments.
| Core theorem | 01e property | role |
|---|---|---|
k_safety' |
conflicting k-finalized blocks ⇒ slashable ⅔ intersection | headline k-accountable safety |
finalization_fork_means_same_finalization_fork_one |
1-fork = same-height k-fork at k=1 | fork-condition bridge |
no_k_finalized_justified_same_height |
no justified block at a finalized height (no slashing) | height uniqueness |
k_slash_surround_case_general |
surround of a finalized block ⇒ slashable | AS surround case |
k_non_equal_height_case |
unequal-height conflicting finalization ⇒ slashable | AS unequal-height case |
slashable_bound |
slashable intersection meets churn-adjusted weight bound | quantitative AS |
quorum_intersection_weight_lower |
⅔-quorum overlap ≥ combined weight − ⅓ thresholds | inclusion-exclusion |
validator_intersection_lower_bound |
validator-set overlap ≥ churn-adjusted expression | weight arithmetic |
no_two_justified_same_height |
no two distinct justified blocks at one height (no slashing) | same-height safety |
two_justified_same_height_slashed |
same-height justification ⇒ slashable | same-height witness |
finalized_means_justified_child |
finalized ⇒ justified child at next height | justification bookkeeping |
k_finalized_means_justified |
k-finalized ⇒ justified | justification bookkeeping |
finalized_means_one_finalized |
finalized ⇔ 1-finalized | definition bridge |
quorum_2_upclosed |
superset of a ⅔-quorum is a ⅔-quorum | quorum monotonicity |
quorum_2_nonempty_of_threshold_pos |
positive threshold ⇒ non-empty quorum | quorum non-emptiness |
plausible_liveness_construct_extension |
honest ⅔ can always extend + finalize without new slashing | plausible liveness (out of bounty scope) |
plausible_liveness_from_coq_blocks_exist |
same, under the Coq block-existence hypothesis | plausible liveness (out of scope) |
no_new_slashed_two_link_extension |
the two-link extension introduces no new slashing | liveness support (out of scope) |
The Executable layer (GasperBeaconChain.Executable.*) is thin and still
growing; its theorems are the decidable/computable Bool checkers for the
Core results, and are mapped as such (S1/S2 slashing checkers,
accountable-safety witnesses, the decidable quorum/justification/liveness
predicates).
The full theorem → implementation-invariant lowering (and the STRIDE class per
property) lives in theorem_map.json.
- Seam: speca#87 (PR #94, merged)
- This work: speca#88
- Downstream repro: speca#92 →
kurtosis-harness