majit: harden generic tracing and diagnostics

[youknowone]

Summary

• separate reusable majit tracing hardening from CEL-specific work
• preserve virtualizable input identity across normal loop and bridge compilation
• harden typed green-key lookup, MIR lowering rollback, resume snapshots, and Cranelift deopt state recovery
• add opt-in opref, register-write, allocation-site, gate-catalog, and trace-shape diagnostics
• compile heavyweight opref and register-write instrumentation only with the non-default jit-audits feature
• add regression coverage for trace shape, gate completeness, lowering, and serialized descriptor behavior

Why

The CEL branch accumulated general JIT infrastructure alongside CEL-specific implementation. Landing the reusable pieces independently reduces the remaining CEL patch while giving the shared tracing and diagnostic machinery focused review and test coverage.

Impact

Ordinary debug and release builds contain no opref or register-write audit modules, call sites, TLS access, environment-gate strings, or report strings. Diagnostic binaries opt in with --features jit-audits; the existing environment variables then select an individual audit. Audit TLS remains limited to disposable per-thread diagnostic state and does not own semantic runtime identity or GC roots.

Validation

• re-extracted majit-rlib, pyre-object, pyre-interpreter, and pyre-jit LLBC; all source fingerprints current
• cargo fmt --all -- --check
• git diff --check
• cargo check --features dynasm
• cargo test --features dynasm
• cargo check -p majit-metainterp --no-default-features --features dynasm
• cargo check -p majit-metainterp --no-default-features --features dynasm,jit-audits
• cargo check -p pyrex --no-default-features --features dynasm,jit-audits
• audit unit tests: 18/18
• gate completeness: 6/6
• trace shape: 2/2
• default release audit symbols and audit strings: 0
• python3 pyre/check.py --backend dynasm --no-synthetic: 17/17

Summary by CodeRabbit

• New Features

  • Added opt-in JIT auditing and diagnostics for operation references, register access, allocation sites, cell tables, and field positions.
  • Added an allocation-attribution example with optional stack capture and summarized reporting.
  • Added clearer reporting for JIT statistics, deoptimization behavior, and performance baselines.

• Bug Fixes

  • Improved bridge compilation handling and typed JIT lookups.
  • Corrected process-creation argument validation and deterministic descriptor ordering.

• Documentation

  • Added comprehensive catalogs of runtime diagnostic settings and retirement criteria.

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 6bd07ee6-d7d6-47d9-8ab3-8db7788ca3e4

📥 Commits

Reviewing files that changed from the base of the PR and between 6fd2f1c and cbf5381.

📒 Files selected for processing (33)

• majit/examples/i64env/src/main.rs
• majit/examples/tinyframe/src/jit_interp.rs
• majit/gate-triage.md
• majit/majit-ir/Cargo.toml
• majit/majit-ir/src/lib.rs
• majit/majit-ir/src/opref_audit.rs
• majit/majit-ir/src/reg_write_audit.rs
• majit/majit-macros/src/jit_interp/jitcode_lower/lower_control.rs
• majit/majit-metainterp/Cargo.toml
• majit/majit-metainterp/src/optimizeopt/mod.rs
• majit/majit-metainterp/src/optimizeopt/optimizer.rs
• majit/majit-metainterp/src/optimizeopt/virtualize.rs
• majit/majit-metainterp/src/pyjitpl.rs
• majit/majit-metainterp/src/pyjitpl/dispatch.rs
• majit/majit-metainterp/src/pyjitpl/frame.rs
• majit/majit-translate/src/codewriter/call.rs
• majit/majit-translate/src/front/mir.rs
• pyre/check.py
• pyre/gate-triage.md
• pyre/pyre-interpreter/src/module/_winapi/mod.rs
• pyre/pyre-jit-trace/build.rs
• pyre/pyre-jit-trace/src/jitcode_dispatch/resume_snapshot.rs
• pyre/pyre-jit/Cargo.toml
• pyre/pyre-jit/src/call_jit.rs
• pyre/pyre-jit/src/eval.rs
• pyre/pyre-jit/src/lib.rs
• pyre/pyre-wasm-runner/src/main.rs
• pyre/pyre-wasm/src/lib.rs
• pyre/pyrex/Cargo.toml
• pyre/pyrex/examples/allocsites.rs
• pyre/pyrex/src/lib.rs
• pyre/pyrex/tests/gate_triage_complete.rs
• pyre/pyrex/tests/jit_trace_shape.rs

---

Walkthrough

The PR adds opt-in JIT audits, bridge identity handling, gate and census diagnostics, allocation profiling, and regression coverage for loop peeling, lowering, translation, and runtime behavior.

Changes

JIT audit instrumentation and bridge handling

Layer / File(s)	Summary
Audit modules and feature wiring majit/majit-ir/Cargo.toml, majit/majit-ir/src/lib.rs, majit/majit-ir/src/opref_audit.rs, majit/majit-ir/src/reg_write_audit.rs, majit/majit-metainterp/Cargo.toml	Adds opt-in OpRef collision and integer-register audits with reporting, attribution, reset, and test controls.
Bridge context and identity resolution majit/majit-metainterp/src/optimizeopt/*	Propagates bridge state into optimization contexts and selects bridge-specific identity slots.
Bridge translation and execution instrumentation majit/majit-metainterp/src/pyjitpl.rs, majit/majit-metainterp/src/pyjitpl/*	Scopes bridge OpRef observations and records integer-register reads, writes, and diverted reads.

JIT lowering and loop regression coverage

Layer / File(s)	Summary
Loop shape and control-flow lowering tests majit/examples/*, majit/majit-macros/src/jit_interp/jitcode_lower/lower_control.rs	Updates peeled-loop expectations and adds failed-lowering state-restoration tests.
Translation and canonicalization regression tests majit/majit-translate/src/codewriter/call.rs, majit/majit-translate/src/front/mir.rs	Tests structural descriptor ordering, cross-block producer resolution, format lowering, and the 176-byte folded layout.

JIT diagnostics and gate validation

Layer / File(s)	Summary
Gate catalog and namespace validation majit/gate-triage.md, pyre/gate-triage.md, pyre/pyrex/tests/gate_triage_complete.rs	Documents live gates and validates Rust, Python, and Cargo environment reads across namespaces.
JIT statistics and vacuity checks pyre/check.py	Merges JIT statistics, rejects missing census denominators, classifies thin baselines, and expands suite reporting.
Runtime and wasm diagnostic plumbing pyre/pyre-jit-trace/*, pyre/pyre-jit/*, pyre/pyre-wasm/*, pyre/pyrex/src/lib.rs, pyre/pyrex/tests/jit_trace_shape.rs	Adds field-position, cell-census, deoptimization, typed-key, mint-index, and trace-shape diagnostics.

Allocation attribution example

Layer / File(s)	Summary
Allocation capture and reporting pyre/pyrex/Cargo.toml, pyre/pyrex/examples/allocsites.rs	Adds a configurable allocator profiler with stack capture, non-vacuity checks, budget handling, and site summaries.

Windows process builtin registration

Layer / File(s)	Summary
CreateProcess arity registration pyre/pyre-interpreter/src/module/_winapi/mod.rs	Registers CreateProcess through the general path with an explicit nine-argument check.

Estimated code review effort: 5 (Critical) | ~120 minutes

Sequence Diagram(s)

sequenceDiagram
  participant JITTrace
  participant BridgeOptimizer
  participant RegisterAudit
  participant JITStats
  JITTrace->>BridgeOptimizer: prepare bridge trace with scoped OpRef records
  BridgeOptimizer->>RegisterAudit: record register reads and writes
  RegisterAudit->>JITStats: emit audit and census totals
  JITStats->>JITStats: merge diagnostics and apply vacuity checks

Loading

Possibly related PRs

• youknowone/pyre#1200: Updates the same peeled-loop regression expectations.
• youknowone/pyre#892: Overlaps with the JIT-statistics parsing and gating changes.
• youknowone/pyre#347: Relates to bridge compilation and peeled-loop behavior.

Poem

I hop through bridges, neat and bright,
And count each register write just right.
Nine loop steps now greet the sun,
While gates report what work was done.
Audit trails make carrots cheer!

✨ Finishing Touches 💡 1

🛠️ Fix failing CI checks 💡

• Create stacked PR
• Commit on current branch

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch cel

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

🤖 Codex parity review

Static analysis of this diff vs the local RPython/PyPy sources (commit cbf5381).
Updated: 2026-08-15T06:48:04.048Z

Files in the reviewed diff

majit/examples/i64env/src/main.rs
majit/examples/tinyframe/src/jit_interp.rs
majit/gate-triage.md
majit/majit-ir/Cargo.toml
majit/majit-ir/src/lib.rs
majit/majit-ir/src/opref_audit.rs
majit/majit-ir/src/reg_write_audit.rs
majit/majit-macros/src/jit_interp/jitcode_lower/lower_control.rs
majit/majit-metainterp/Cargo.toml
majit/majit-metainterp/src/optimizeopt/mod.rs
majit/majit-metainterp/src/optimizeopt/optimizer.rs
majit/majit-metainterp/src/optimizeopt/virtualize.rs
majit/majit-metainterp/src/pyjitpl.rs
majit/majit-metainterp/src/pyjitpl/dispatch.rs
majit/majit-metainterp/src/pyjitpl/frame.rs
majit/majit-translate/src/codewriter/call.rs
majit/majit-translate/src/front/mir.rs
pyre/check.py
pyre/gate-triage.md
pyre/pyre-interpreter/src/module/_winapi/mod.rs
pyre/pyre-jit-trace/build.rs
pyre/pyre-jit-trace/src/jitcode_dispatch/resume_snapshot.rs
pyre/pyre-jit/Cargo.toml
pyre/pyre-jit/src/call_jit.rs
pyre/pyre-jit/src/eval.rs
pyre/pyre-jit/src/lib.rs
pyre/pyre-wasm-runner/src/main.rs
pyre/pyre-wasm/src/lib.rs
pyre/pyrex/Cargo.toml
pyre/pyrex/examples/allocsites.rs
pyre/pyrex/src/lib.rs
pyre/pyrex/tests/gate_triage_complete.rs
pyre/pyrex/tests/jit_trace_shape.rs

1. Regressions to PyPy parity introduced by this patch

None.

2. Other mismatches introduced by this patch

None.

3. Pre-existing mismatches (already present before this patch)

• pyre/pyre-jit/src/eval.rs:5606 ↔ pypy/module/pypyjit/interp_jit.py:84 — pyre hard-codes is_being_profiled to false when constructing the portal green key; PyPy reads the frame’s profiling state and includes it in the green tuple. The patch preserves this existing collapse while fixing typed cell lookup.

• majit/majit-metainterp/src/optimizeopt/virtualize.rs:147 ↔ rpython/jit/metainterp/pyjitpl.py:1122 — pyre retains virtualizable field state in an optimizer-side VirtualizableTracker; RPython keeps the standard virtualizable and its field boxes on MetaInterp.virtualizable_boxes during tracing. This tracker predates the patch; the patch only corrects its phase-2/bridge identity-slot discrimination.

4. Structural adaptations

None.