Skip to content

JIT: hot loops inside a with-block / comprehension never receive a JIT token (no-token cliff behind #373) #389

Closed
Closed
JIT: hot loops inside a with-block / comprehension never receive a JIT token (no-token cliff behind #373)#389
#490 (+2)

Description

@youknowone
youknowone
opened on Jul 5, 2026

Status (2026-07-22, merged to main) — CLOSED. All three items resolved; the append-fold virtualization now ships default-on and the deferred rooting slices are retired as dead machinery.

Since the 2026-07-18 note below, the branch rebased onto main and pulled in #344 (retire the trace-time single-executor forks — every production walk is now a full-body walk). That structurally removed the speculative-replay sub-walk the whole "root the nested-comprehension inner-list backing block across a deopt" epic (slices S2a / S2 / S2.2 / S2c) existed to serve: the inner [[i] …] backing block is now bound at every guard-exit deopt without any extra resume-data root. Consequences:

⇒ #389's three cliffs are all resolved and the follow-on virtualization is shipped. Closing. Any wasm-backend deopt/resume follow-up for the unboxed append fold (currently gated off on wasm via PYRE_WASM_UNBOXED_APPEND_FOLD) is a wasm-backend bug, not #389 core.

Status (2026-07-18, no-token) — ALL THREE items now COMPILE with correct results; (b) resolved for call-free comprehension bodies, the remaining sub-case is an adjudicated 2-executor architectural limitation.

Two commits on origin/no-token (037341694a7, cdda4794857), each zero-regression on both backends (check.py 217/217 dynasm + cranelift):

  • (a) in-loop MAKE_FUNCTION — ✅ RESOLVED (unchanged). Repro loops_compiled=2, result correct.

  • (c) with cliff — ✅ RESOLVED (jit: drop the WITH_EXCEPT_START gate + with-frame exception-correctness fixes (#389) #532, unchanged). Repro loops_compiled=2, result correct.

  • (b) inlined-comprehension [j for j in range(n)] — ✅ RESOLVED (compiles). The issue's exact repro ([j for j in range(10)] in while<3000) now returns 30000 with loops_compiled=2; the standalone [i for i in range(1000)] inner loop compiles loops_compiled=1 loops_aborted=0 (was 0). Two root causes fixed:

    1. 037341694a7 — operand-stack mirror corruption at the comprehension FOR_ITER-entry permutation. The py3.14 inlined comprehension emits a SWAP/BUILD_LIST/SWAP preamble before its FOR_ITER, which the codewriter lowers non-monotonically in jitcode; the JitCode-PC floor pivot maps a later jit_pc back to an earlier py_pc, so reconcile_vstack_at_boundary re-applied the SWAP effect and reused a stale vstack_last_ref across the re-walk, dropping the just-built list from the vstack_boxes mirror. The stack_sync overlay then copied the corrupt mirror into the resume snapshot, so a deopt reconstructed a frame whose FOR_ITER iterator slot held a duplicated iterator and a dropped list ("not an iterator"). Fix: detect the backward (non-monotonic) py transition after a Swap/Copy and reseed the mirror from the virtualizable shadow (ShadowReseed) across the permutation region.
    2. cdda4794857 — LIST_APPEND list receiver unbound inside the resident loop. The codewriter lowered LIST_APPEND's list operand by cloning the compile-time FlowValue, but the comprehension accumulator lives on the operand stack, which is not loop-carried in registers (jit_merge_point reds are [frame, ec]), so inside the resident loop that slot's register was unbound → the jit_list_append residual read an unbound receiver and aborted ResidualCallArgUnbound before ever reaching the #171 append fold (so the comprehension inner loop never compiled and the empty-append fold was dead). Fix: reload the list receiver from its value-stack slot via getarrayitem_vable_r each iteration, the same per-iteration re-materialization the FOR_ITER iterator reload already uses.

    Remaining sub-case (guarded, not a correctness bug): a comprehension whose body enters a user Python frame, e.g. [C(i) for i in range(n)]. Such a body's per-element call sets the FOR_ITER in-flight body-effect flag; a mid-body abort then routes through fbw_foriter_inflight_take, which refuses in-flight delivery to avoid a double-apply, dropping the trace-attempt iteration's item. for_iter_bodies_all_jit_safe admits LIST_APPEND only when its FOR_ITER body performs no call, so this shape declines cleanly to the interpreter (byte-correct, un-compiled). Empirically, force-delivering the item makes it correct — the refusal is the sole blocker — but delivery is unsafe in general (a body with a genuine external mutation would double on re-run), and pyre cannot cheaply distinguish C.__init__'s write-to-fresh-self from a global-mutating call without the object virtualization PyPy has at the optimizer layer. This is a genuine pyre 2-executor (observer/replay) limitation, deferred to single-executor tracing (Eliminate pc_map and per-PC labels by switching the tracer to interpret JitCode bytecode #73/Fix merge points in fannkuch loop #34) or object virtualization — not an append-fold/gate tweak. Result stays byte-correct throughout.

Earlier status notes (superseded, kept for history)

Status (2026-07-15, later update — on the no-token branch atop the rewrite-tracer base where #73 is landed) — items (a) and (c) RESOLVED (#532, MERGED ef7d2089134); (b) advanced through THREE walker layers, now narrowed to a single sub-walk epic. The three landed layers are pushed to origin/no-token (7fdca719f50), each zero-regression on both dynasm and cranelift (check.py 187 pass / 1 pre-existing BASEFAIL synth/list_insert_pop_index).

  • (a) in-loop MAKE_FUNCTION/SET_FUNCTION_ATTRIBUTE — ✅ RESOLVED. Per-iteration def repro compiles: loops_compiled=1 loops_aborted=0. Result correct.

  • (c) with cliff — ✅ RESOLVED by jit: drop the WITH_EXCEPT_START gate + with-frame exception-correctness fixes (#389) #532 (drop the WITH_EXCEPT_START gate). With-loop repro compiles loops_compiled=2. The test_shlex regression the gate-drop exposed is fixed inside jit: drop the WITH_EXCEPT_START gate + with-frame exception-correctness fixes (#389) #532 (read_token mapdict LOAD_ATTR fold dropped the promoted-map guard_value → wrong storage slot; fixed by emitting the map guard unless the map op is already constant).

  • (b) inlined-comprehension [j for j in range(n)] — PARTIALLY ADVANCED; now blocked on the sub-walk epic, NOT the operand-materialization gap the prior note named. On the rewrite-tracer base the outer hot loop containing an inlined comprehension IS admitted and traced (no longer declined at the FOR_ITER user-defined iterator: __next__ inlining blocked by frozen root-frame concrete and threshold alignment #57 frame gate); the walk reaches the comprehension's L5 epilogue. Three layers landed, each moving the abort strictly forward and each independently zero-regression:

    1. a4106b87199 — classify LIST_APPEND/SET_ADD/MAP_ADD/LIST_EXTEND as PopOnlyOrSideStore in the walk-level operand-stack mirror (classify_vstack_opcode), so the mirror stays valid across the comprehension loop instead of latching vstack_valid=false. Moved the abort from ResidualCallArgUnbound → JitMergePointGreenKeyUnresolved.
    2. b546b4f2be8 — forward-select the nested inner-loop header merge point (delete header_entry_is_nested), so the comprehension's inlined FOR_ITER gets a concrete pycode green instead of OpRef::NONE. Moved the abort to GuardSnapshotVableUntyped + UnfoldableListAppendResidualUnsupported.
    3. 7fdca719f50 — recover a nested-loop-carried STORE_FAST value from the vstack mirror. RCA (probe-confirmed): the L5 epilogue STORE_FAST xs / STORE_FAST j lower to setarrayitem_vable_r whose value register reads OpRef::NONE — xs (the BUILD_LIST accumulator) and saved j (LOAD_FAST_AND_CLEAR) are operand-stack temps that span the inner FOR_ITER, and the loop-header re-seed seeds no operand-stack colors, so their registers are lost. The value survives ONLY in the vstack mirror TOS (InputArgRef), not in the vable shadow (cleared to ConstPtr(0)). Reading the mirror TOS instead of the stale register avoids writing an untyped box into the virtualizable array (which failed the guard-snapshot buildability precondition, GuardSnapshotVableUntyped). Notably this was NOT the #124 multi-frame-vable case the earlier note guessed: the identity slot was typed; two operand-stack-temp array slots were OpRef::NONE.

    Remaining wall = UnfoldableListAppendResidualUnsupported → the sub-walk guard-side-exit resume epic (Align flatten_graph to pypy #62/Eliminate pc_map and per-PC labels by switching the tracer to interpret JitCode bytecode #73/Fix merge points in fannkuch loop #34), NOT an append-fold/gate tweak (probe-confirmed, superseding the prior "blocked on operand materialization / Eliminate pc_map and per-PC labels by switching the tracer to interpret JitCode bytecode #73 seeding" framing). With the three layers in place the sole remaining abort is the list-append fold declining. Drilled with staged experiments (all reverted): the comprehension's FIRST traced append is strategy=Empty, value=int, so two barriers stack — (1) the Empty→Integer strategy SWITCH (the #171 fold only recognizes already-typed Integer/Float/Object storage; Empty matches none), and (2) the first alloc (Empty has no backing array). Bypassing the recognize gate to admit Empty+int makes the commit sub-walk abort OrthodoxSubWalkTraceUnsupported: descending the w_list_append body hits switch_to_correct_strategy + grow_list_items_block (#[dont_look_inside]), whose in-body helper carries a symbolic_fnaddr (>>47 hash, e.g. the SyntheticTransparentCtor "Tuple" unit ctor) that a compiled sub-walk trace must not bake as a code address (→ SIGSEGV). So (b) is genuinely gated behind either (A) a realloc-safe append journal / retiring the legacy replay-from-snapshot rollback (the FBW_APPEND_JOURNAL is pure-length-rewind only; a realloc frees/relocates the block irreversibly), or (B) a correct sub-walk residual + guard-side-exit reconstruction for the dont_look_inside grow. Being on the rewrite-tracer base (Eliminate pc_map and per-PC labels by switching the tracer to interpret JitCode bytecode #73 landed) did not dissolve either wall — they are downstream of Eliminate pc_map and per-PC labels by switching the tracer to interpret JitCode bytecode #73. Multi-session; correctly deferred. Result stays byte-correct (30000) throughout — (b) is a missing-speedup cliff, never a correctness bug.

Correction (2026-07-08, second refresh) — the first refresh note above mis-diagnosed the comprehension case; this supersedes it.
Re-measured on a clean build at the current tip (after #344 Phase B / #426 / #427). Two claims in the first refresh note are wrong and are corrected here.

Correction 1 — pyre DOES inline comprehensions (PEP 709). The first note claimed "RustPython compiles a comprehension as a separate <listcomp> code object (no PEP 709 inlining)". That is false: [sys._getframe().f_code.co_name for _ in range(1)] returns 'main', not '<listcomp>' — the same as CPython ≥ 3.12 and pyre's 3.14 compat target. This is interpreter-level (holds under PYRE_JIT=0), so it is not a JIT artifact. The listcomp's FOR_ITER + LIST_APPEND are inlined directly into the enclosing frame.

Correction 2 — the two repros have DIFFERENT root causes; the 0/5 abort does not belong to the comprehension. The first note conflated them. Measured separately:

  • comprehension [j for j in range(10)] in a hot loop → loops_compiled=0 loops_aborted=0, main never starts tracing (0 start tracing lines even at 200k iters). Because the listcomp is inlined, main is a FOR_ITER frame whose body contains LIST_APPEND, which is not in the for_iter_body_op_is_jit_safe allow-list (eval.rs). So unsupported_jit_shape classifies main as CurrentFrameOnly and declines it before the counter ticks (eval_with_jit_inner) — the FOR_ITER user-defined iterator: __next__ inlining blocked by frozen root-frame concrete and threshold alignment #57 non-journalable-mutator soundness gate. Direct control: an explicit inlined for j in range(10): out.append(j) in the hot loop (no comprehension, no def) declines identically (traces_started=0) — confirming the comprehension is incidental; the cause is the inlined-append FOR_ITER user-defined iterator: __next__ inlining blocked by frozen root-frame concrete and threshold alignment #57 gate.

  • a per-iteration inner def whose body is for+list.append (the first note's "minimal repro") → loops_compiled=0 loops_aborted=5, and main does start tracing then aborts. But the abort is not "walker fails to residualize the declined callee": a per-iteration def that is defined but never called still aborts 0/5. The real cause is MAKE_FUNCTION → emit_abort_permanent! (codewriter.rs:9280, and its partner SET_FUNCTION_ATTRIBUTE at :10647), which lands after the loop header jit_merge_point, so the up-front loop_body_has_abort_permanent scan (trace.rs:1638) declines the whole outer loop. The trait tracer already residualizes this op (trace_make_function → jit_make_function_from_globals, helpers.rs:558); only the full-body-walk codewriter aborts. The named-module-helper control (identical loop, def hoisted out) compiles (loops_compiled=1), isolating the in-loop MAKE_FUNCTION as the sole difference.

Revised remaining work (three independent items, all correctness-neutral cliffs — every repro produces the correct result today):

The original "invisible cliff" framing (tracer never starts / empty census / gate upstream) was already corrected in the first note and remains fixed by PR #410. Everything below is the original report, kept for history.

Refresh (2026-07-08, after PR #410) — the original diagnosis below is superseded.
The "no-token cliff" as originally framed ("the tracer never starts; loops_compiled=0 and loops_aborted=0; empty FBW census; the gate is upstream of the tracer") no longer holds. PR #410's codewriter walker fall-through-into-joinpoint fix removed the invisibility: the tracer now does start on both repros, reaches the frame-shape gate, and declines visibly (non-empty census). Re-measured on the current no-token tip:

with case — loops_compiled=0 but census is now populated: FrameShape::StructuralRegion + FrameShape::CurrentFrameOnly entries. This is the restored StructuralRegion gate (S1 was reverted: removing it miscompiled with-frames — SIGSEGV in the exception path + guard storms, the PR #410 CI regression). The loop declines cleanly because its WITH_EXCEPT_START exception-link lowering is still residualized. Compiling it needs the exception-link lowering — that is #373 work, correctly deferred here.

comprehension case — loops_compiled=0 loops_aborted=5, census shows FullBodyWalk::LoopBodyAbortPermanent: 5 + Trait::DeclinedAbort: 5 on the outer main loop, and FrameShape::CurrentFrameOnly on the comprehension frame. Isolation (repros below) pins the real cause: it is not "the tracer never starts" and not the comprehension per se. RustPython compiles a comprehension as a separate <listcomp> code object (no PEP 709 inlining) containing FOR_ITER + LIST_APPEND → the #57 non-journalable-mutator gate declines that callee as CurrentFrameOnly (correct). The remaining defect is that the outer while loop aborts (abort_permanent) when the walker tries to inline that declined callee, instead of emitting it as a residual call and compiling the outer loop. This is the same family as task#38 (walker aborts on a callee it should residualize). Minimal reproduction that is not a comprehension but reproduces 0/5 exactly:

def main():
    total = 0; i = 0
    while i < 3000:
        def lc():                 # per-iter MAKE_FUNCTION + FOR_ITER + list.append
            out = []
            for j in range(10):
                out.append(j)
            return out
        xs = lc()
        total += len(xs)
        i += 1
    print(total)
main()

Controls that isolate it (all on the current tip):

  • while loop, no call → compiles (loops_compiled=1).
  • while loop calling a plain helper() returning a constant → compiles.
  • while loop calling a named helper with for+list.append → compiles (loops_compiled=1): a declined-mutator callee reached via a named function is residualized fine.
  • while loop with a per-iteration inner function (or comprehension) whose body is for+list.append → loops_compiled=0 loops_aborted=5: the walker aborts the outer loop instead of residualizing.

Revised remaining work: (a) comprehension/while cliff = make the walker residualize a declined callee and keep compiling the outer loop (task#38 family — the walker's abort-on-uncompilable-callee should be a residual, not an outer-loop abort_permanent); (b) with cliff = the WITH_EXCEPT_START exception-link lowering (#373). The invisibility bug (original framing) is fixed by PR #410. Everything below is the original report, kept for history.

Summary

A hot loop whose body sits inside a with block (or another
exception-table / block-stack region), and a hot loop that rebuilds a
comprehension each iteration, never receive a JIT token at all. The tracer does
not start, no loop is recorded, and the loop runs in the pure interpreter forever.

This is a distinct failure mode from both:

Here the tracer is never even invoked — loops_compiled=0 and loops_aborted=0,
with an empty FBW census (the abort_permanent up-front decline is never reached). It is
the "no-token cliff" that masks the latent-but-portable abort_permanent opcodes noted in
#373 (WITH_EXCEPT_START, LOAD_SPECIAL Enter/Exit, the comprehension accumulators):
their ports are correct, but no bench can demonstrate a win because the loops that contain
them never compile in the first place.

Reproduction (dynasm/cranelift, 84261b23a5)

with block in a hot loop — never compiles:

class CM:
    def __enter__(self): return self
    def __exit__(self, *a): return False

def main():
    total = 0
    i = 0
    while i < 200000:
        with CM():
            total += 1
        i += 1
    print(total)

main()
$ MAJIT_STATS=1 pyre-dynasm with_loop.py
200000
[jit-stats] loops_compiled=0 bridges_compiled=0 loops_aborted=0 guard_failures=0 internal_compile_panics=0

Control — identical loop without with — compiles:

def main():
    total = 0
    i = 0
    while i < 200000:
        total += 1
        i += 1
    print(total)
main()
[jit-stats] loops_compiled=1 bridges_compiled=0 loops_aborted=1 guard_failures=1 internal_compile_panics=0

Comprehension rebuilt each iteration — never compiles:

def main():
    total = 0
    i = 0
    while i < 3000:
        xs = [j for j in range(10)]
        total += len(xs)
        i += 1
    print(total)
main()
[jit-stats] loops_compiled=0 bridges_compiled=0 loops_aborted=0 guard_failures=0 internal_compile_panics=0

Evidence that the tracer never starts

  • MAJIT_STATS: both cliff cases show loops_compiled=0 and loops_aborted=0. An
    aborted trace would bump loops_aborted; a declined abort_permanent body would show
    in the FBW census. Neither happens.
  • PYRE_FBW_DEBUG_ABORT=1 census is empty for both cases — the abort_permanent
    up-front decline (FullBodyWalk::LoopBodyAbortPermanent, trace.rs) is never reached.
  • MAJIT_LOG=1 prints zero start tracing / loop_header / jit_merge_point lines
    for the with case, versus one start tracing for the identical with-free control.

So the gate is upstream of the tracer and of the abort_permanent decline: the hot loop
inside the exception-table region is never detected as a compilation candidate (or its
can_enter_jit back-edge is never counted toward the compile threshold).

emit_loop_header (the lowered can_enter_jit, codewriter.rs:2224, emitted
unconditionally at each JumpBackward, codewriter.rs:7994-8015) is present in the
jitcode, so the miss is not "no loop_header op emitted". The next suspect is the runtime
hot-loop threshold accounting (warmstate) or how a back-edge inside a nonempty block-stack
region (blocklist, codewriter.rs:261) is treated — root cause to be pinned in
investigation.

Why it matters

  1. with blocks and comprehensions are common in real Python; any hot loop using them
    silently falls off the JIT.
  2. It blocks closing JIT: unported opcode (CALL_INTRINSIC_1 etc.) in a hot loop body declines the whole loop → never compiles #373: the latent-but-portable opcodes (WITH_EXCEPT_START,
    LOAD_SPECIAL, SET_ADD/MAP_ADD/DICT_MERGE and friends) are ported and correct,
    but this cliff prevents any hot loop containing them from ever compiling, so the ports
    cannot be demonstrated and JIT: unported opcode (CALL_INTRINSIC_1 etc.) in a hot loop body declines the whole loop → never compiles #373 cannot be fully verified against real loops.

Investigation / fix path

  1. Pin the exact gate: instrument the warmstate hot-loop counter / can_enter_jit
    back-edge accounting for a back-edge whose blocklist is nonempty (inside a with /
    try region), and for the comprehension inner function. Determine whether the back-edge
    is never counted, counted under a different green key, or the candidate is filtered
    before tracing.
  2. Compare against RPython can_enter_jit / jit_merge_point semantics for a
    loop nested inside a block-stack region — RPython compiles these.
  3. Once a loop inside a with compiles, re-check that the ported WITH_EXCEPT_START /
    LOAD_SPECIAL residuals behave (they were only latent-verified in JIT: unported opcode (CALL_INTRINSIC_1 etc.) in a hot loop body declines the whole loop → never compiles #373).

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions