FBW inline sub-walk concrete-residual double-apply / value-corruption (bug class, #177 one frame deeper)

[youknowone]

The FBW inline sub-walk (multiframe inline capture, #68) executes residual CALLs concretely inside a walk that can still abort and rewind. When such a residual commits a heap mutation (or runs user Python bytecode that does) and the sub-walk then hits a permanent abort, the rollback (which undoes only the 3 FBW journals: store / append / cell) misses the concrete write, and the interpreter re-runs the iteration — the effect applies twice, a delivered value is corrupted, or it crashes. This is #177 one frame deeper, and it is a bug class, not a single defect.

Upstream parity framing: RPython MetaInterp records residual CALLs as ops and replays them forward by the compiled bridge; it never executes a residual during resume/rebuild (rebuild_from_resumedata is pc-assign only; a deep RETURN propagates its value up via finishframe/_resume_mainloop, never re-entering the callee). The deviation is: pyre executes a residual concretely inside a walk that can rewind.

Orthodox invariant to restore: a concretely-executed residual must never be re-executed. Either (a) the inline capture is declined before the concrete write (the write then happens exactly once, in the outer/interpreter context), or (b) the walk commits its executed region and the interpreter resumes strictly after it (forward-only resume, never rollback+replay).

A narrow fix landed on single-walker (jit: decline unjournaled mutating residual executed inside an inline sub-walk, currently 8bb1e98d1c2, unpushed): a pre-call decline in try_execute_residual_call_via_executor gated on the writes_live_heap discriminator (Void result or CallFn/StoreSubscr/SetCurrentException/StoreDeref helper tag). It closes only the narrowest slice (Void-tagged store, single inline depth, branch-not-taken abort). 7 distinct defects survive, verified on the HEAD binary (all rows default (JIT) != oracle (PYRE_JIT=0)), in two families.

Family A — #68 inline-multiframe capture path (PYRE_FBW_INLINE_MULTIFRAME=0 fixes)

Same mechanism as the landed fix; the decline misses these because a Ref-result residual (a @property/__add__/__radd__/__getattr__ getter returns a value) carries writes_live_heap=false, and the user-frame odometer that could catch it (user_frame_snapshot/entered_user_frame) is gated on fbw_foriter_inflight_active() — it only marks the FOR_ITER delivery path, so in a while-loop inline capture it is None.

repro	oracle	default	defect
hunt2_3_l.py (@property mutating getter + try/except in callee)	60000	60005	over-mutate
hunt2_2_b.py (__radd__, branch consumes the value)	(2, 40000)	(6, 40001)	delivered value corrupted
hunt_user_iterator_in_callee_3_forloop.py (user-iterator for inside inlined callee)	(96000, 54000)	PANIC	crash majit-backend/src/call_stub.rs:442 (BhCallDescr arity)

hunt2_3_l.py

# @property value-returning mutating + try/except-inside-callee raising branch
N = 60000
class C:
    def __init__(self): self.pos = 0
    @property
    def tick(self):
        self.pos = self.pos + 1
        return self.pos
def step(c, d, k):
    t = c.tick
    if k < 0: return 0
    try:
        return d[k]
    except KeyError:
        return -1
def run():
    d = {1:100,2:200,3:300}; acc=0; c=C(); i=0
    while i < N:
        k = i % 5
        v = step(c, d, k)
        if v == -1: acc -= 1
        else: acc += v
        i += 1
    return acc, c.pos
print(run())

hunt2_2_b.py

# radd result feeds the branch condition; branch varies each iter -> sub-walk churn
N = 40000
class Acc:
    def __init__(self): self.pos = 0
    def __radd__(self, o):
        self.pos = self.pos + 1
        return self.pos + o
def step(c, k):
    t = k + c                 # c.__radd__ residual, returns int depending on pos
    if t & 1: return 1        # branch depends on mutating residual result
    if k < 0: return 0
    return -1
def run():
    acc=0; c=Acc(); i=0
    while i < N:
        v = step(c, i % 7)
        acc += v
        i += 1
    return acc, c.pos
print(run())

hunt_user_iterator_in_callee_3_forloop.py

# FOR loop over user iterator INSIDE branch-bearing inlined callee; __next__ mutates shared counter
N = 30000
class It:
    def __init__(self): self.pos = 0; self.lim = 3
    def __iter__(self):
        self.n = 0
        return self
    def __next__(self):
        self.n = self.n + 1
        if self.n > self.lim:
            raise StopIteration
        self.pos = self.pos + 1
        return self.n
def step(it, d, k):
    if k < 0:
        return 0
    if k in d:
        s = 0
        for x in it:          # FOR_ITER over user iterator inside inlined callee branch
            s += x
        return s
    return -1
def run():
    d = {1:100,2:200,3:300}; acc=0; it=It(); i=0
    while i < N:
        k = i % 5
        v = step(it, d, k)
        acc += v
        i += 1
    return acc, it.pos
print(run())

Family B — NOT the inline-multiframe path (PYRE_FBW_INLINE_MULTIFRAME=0 does NOT fix)

Status update (localized): Family B is two distinct defects.

B1 (rows 1-2) — top-level walk-abort double-apply. run_perfn_walk (trace.rs:1147) concretely executes a mutating residual whose heap write happens inside the concrete Python call (never enters FBW_STORE_JOURNAL); the walk then aborts — RunPerfnWalkNone (trace.rs:2132) for the depth-3 chain, Terminate::NoFinishPayload (trace.rs:2032, exception-driven ungated portal exit) for mutate-then-raise — and the post-abort replay from the recorded entry state re-runs the mutation. These are top-level occurrences (FBW_INLINE_CODE_STACK empty), so the Phase-0 decline never fires: its premise "at top level the legacy-replay path is sound" is false for an effect committed inside a concrete residual call. Evidence: PYRE_FBW_NESTED_RESID_ABORT=0 makes both worse (112039 / 30010 — the decline is helping where it does fire, but has this coverage gap); PYRE_FULL_BODY_WALK=0 fixes both; the +1 is N-invariant (one-time trace-install event); the depth-3 minimization triad shows the doubled write is levelB's bump() with the abort triggered by its subsequent residual call, and the mutate-then-raise variant is correct without the raise.

B2 (rows 3-4) — inlined-closure freevar-read + branch miscompile (NO abort, not double-apply). The walk succeeds and installs a miscompiled trace: an inlined closure that both reads a freevar (LOAD_DEREF) and has a data-dependent branch gets the freevar box aliased into the branch-arm return/resume slot (int freevar → return value corrupted by exactly the freevar's value; list freevar → the return int is written into the cell slot, smashing the list → TypeError on the next iteration). Mutation is not required — a read-only freevar + branch reproduces (minimal repro below: 210 vs oracle 1600000). nonlocal in row 3 was a red herring. Tracked separately as #498.

repro	oracle	default	defect
hunt_nested_depth_chain_1.py (depth-3 mutating chain)	(…, 112000)	(…, 112001)	B1 over-mutate
hunt_exception_variants_2.py (mutate-then-raise, caught in callee)	30000	30005	B1 over-mutate
hunt_global_nonlocal_mutation_LEAK.py (nonlocal StoreDeref in inlined closure)	acc 1600000	acc 40198	B2 return value corrupted
hunt_global_nonlocal_mutation_6_nonlocal_list.py (freevar list in inlined closure)	(1600000, 40000)	TypeError	B2 crash (freevar list-cell smashed to int)

v4_readonly_branch.py (minimal B2 repro — read-only freevar, no mutation)

# LOAD_DEREF read-only freevar (no append/store) + branch
N = 40000
def run():
    base = 10
    acc = 0
    def step(k):
        if k == 2: return base + 190
        return -1
    i = 0
    while i < N:
        v = step(i % 5)
        acc += (v if v != -1 else 0)
        i += 1
    return acc
print(run())

oracle 1600000, default JIT 210 (one correct pre-trace contribution + the freevar's value leaked once; scales with base).

hunt_nested_depth_chain_1.py

# 3-level nested branch-bearing mutating callees. outer while -> A -> B -> C, each mutates.
N = 40000

class Counter:
    def __init__(self):
        self.pos = 0
    def bump(self):
        self.pos = self.pos + 1

def levelC(c, d, k):
    c.bump()
    if k < 0:
        return 0
    if k in d:
        return d[k]
    return -1

def levelB(c, d, k):
    c.bump()
    if k == 4:
        return 7
    return levelC(c, d, k)

def levelA(c, d, k):
    c.bump()
    if k < 0:
        return -3
    return levelB(c, d, k)

def run():
    d = {1: 100, 2: 200, 3: 300}
    acc = 0
    c = Counter()
    i = 0
    while i < N:
        k = i % 5
        v = levelA(c, d, k)
        if v == -1:
            acc -= 1
        else:
            acc += v
        i = i + 1
    return acc, c.pos

print(run())

hunt_exception_variants_2.py

# V2: mutate-then-RAISE, caught inside callee. Tests fixed path under exc churn.
N = 30000
class C:
    def __init__(self):
        self.pos = 0
    def tick(self):
        self.pos = self.pos + 1
        raise ValueError
def step(c, d, k):
    try:
        c.tick()             # void residual mutating then raising
    except ValueError:
        pass
    if k < 0:
        return 0
    if k in d:
        return d[k]
    return -1
def run():
    d = {1:100,2:200,3:300}; acc=0; c=C(); i=0
    while i < N:
        k = i % 5
        v = step(c, d, k)
        acc += (v if v != -1 else 0)
        i += 1
    return acc, c.pos
print(run())

hunt_global_nonlocal_mutation_LEAK.py

# nonlocal StoreDeref (n+=1) before branches in a loop-free inlined callee,
# driven by a hot while-loop. Mutation counter n is CORRECT but the callee's
# branch RETURN VALUE is corrupted under the default JIT -> acc diverges.
N = 40000
def run():
    n = 0
    acc = 0
    def step(k):          # inlined (call not in try, no loop in callee)
        nonlocal n
        n = n + 1         # StoreDeref before the branches
        if k < 0:
            return 0
        if k == 2:
            return 200
        return -1
    i = 0
    while i < N:
        k = i % 5
        v = step(k)
        acc += (v if v != -1 else 0)
        i += 1
    return acc, n
print(run())

hunt_global_nonlocal_mutation_6_nonlocal_list.py

N = 40000
def run():
    buf = []
    acc = 0
    def step(k):
        buf.append(k)          # LOAD_DEREF buf + list.append residual (no StoreDeref)
        if k < 0:
            return 0
        if k == 2:
            return 200
        return -1
    i = 0
    while i < N:
        v = step(i % 5)
        acc += (v if v != -1 else 0)
        i += 1
    return acc, len(buf)
print(run())

Staged plan

• Phase 0 — correctness floor (Family A): ✅ LANDED (769af430e3b, single-walker): the pre-call decline in try_execute_residual_call_via_executor generalized from writes_live_heap && !provably_side_effect_free to !provably_side_effect_free — any non-provably-pure residual inside an FBW inline sub-walk is declined before it runs (the decline helper no-ops unless FBW_INLINE_CODE_STACK is non-empty, so top-level depth-1 is untouched). Verified: all 3 Family A repros now match the oracle (including the call_stub.rs:442 crash), adv_3_a/b stay fixed, check.py --backend dynasm 161/161. Cost: forfeits inline capture for non-pure residuals inside a sub-walk — a perf regression, not a correctness one.

• Phase 1 — restore the inlining (redefined): the original forward-commit-at-abort sketch is not implementable — at the mid-callee permanent abort no interpreter frame exists for the callee (the sub-walk is purely symbolic, and flush_walk_end_state_to_frame is single-frame/outer-pc keyed). Replacement: eliminate the abort at its source. step_vstack_mirror invalidates the operand-stack mirror the moment an inline sub-walk starts (callee jitcode pcs don't exist in the outer py_pc tables), which forces !vstack_valid → BranchGuardUnrestorableKeptStackPermanent for every branchy callee; the mirror_covers_kept bypass already compiles kept-stack guards when the mirror is valid and covering. Reconciling the mirror across the callee-coordinate boundary lets branchy callees sub-walk to SubReturn, where the existing outer end-flush commits forward with the residual counted exactly once. Phase 0 stays as the floor for genuinely un-recordable callees.

• Phase 2 — B1 walk-abort double-apply: ✅ LANDED (5 commits on single-walker, verified default == oracle for every B1 repro + minimized variants, incl. under PYPY_GC_NURSERY=65536/131072; check.py --backend dynasm 161/161 at each step):

  • 9c976880a20 — FBW_EXECUTED_NONPURE_RESIDUAL latch (set at the residual executor on both Ok and Err when not provably side-effect-free; reset at walk entry).
  • 6456fb5031c — Stage 1: int_return/c and float_return/f now stash the concrete result like the ref/int/void return arms, so the terminate_no_replay no-replay exit covers every value-returning function.
  • 88c2f4e3a42 — Stage 2: uncaught exception exits (raise/r, reraise/, SubRaise exhaustion) stash the concrete exception (GC-rooted) and forward-deliver it — TraceAction::Finish { exit_with_exception } + eval-side Err return — instead of the Terminate::NoFinishPayload → abort → entry-replay that doubled the pre-raise mutation. Opt-out: PYRE_FBW_RAISE_NO_REPLAY=0.
  • 8e8039a666f — Stage 3: LoopBearingCalleeInlineUnsupported aborts (a nested inline decline after the walk already executed a non-pure residual) forward-commit via the abort-flush leg. The error pc is a callee coordinate, so the caller CALL py_pc + concrete operand-stack slots are stashed at inline-capture time and the flush uses those (flush_walk_end_state_to_frame_with_stack_overrides); the interpreter resumes at the CALL and runs the declined callee exactly once. Opt-out: PYRE_FBW_ABORT_FLUSH=0.
  • c35de3fb255 — GC-root the stack-override stashes (raw refs held across residual-executing windows; same class as the store-journal root walker).

  Known residual debt: 88c2f4e3a42 also converts the CatchExceptionWithActiveException invariant abort into a conditional clear-and-continue (gated on the latch + the kill-switch) — this closes the remaining replay aborts of the mutate-then-raise shape but is a workaround; the root question (what leaves last_exc_value set after in-frame handling) is still open.

• Phase 3 — B2 inlined-closure freevar+branch miscompile: tracked in JIT: inlined-closure freevar read + branch miscompile — freevar cell aliases the branch-arm return slot (wrong value / TypeError crash) #498 (not double-apply; independent trace-correctness defect; minimal repro above).

Hazard / ordering constraint: do not relax any other inline decline site (the vstack-mirror permanent abort, the try-block decline, the !is_multiframe_resume gate) before the forward-commit lands — relaxing first converts the bounded trace-time leak into an unbounded runtime frames>1 double-apply.

Regression oracle set — LANDED in the synth parity suite (pyre/bench/synth/, commit 0ca576f9397, check.py --backend dynasm now 171/171): inline_subwalk_mutating_residual_abort / _noexc (adv_3_a/b), inline_subwalk_property_mutates, inline_subwalk_radd_consumed, inline_subwalk_user_iterator, nested_callee_chain_mutation_abort, mutate_then_raise_caught, float_return_side_effect, const_int_return_side_effect, mutate_uncaught_raise_delivery. The B2 repros (#498) are not in the suite — they still fail; add them with the #498 fix.

— commented by Claude

---

EPIC FINAL STATUS (2026-07-12, single-walker @ b34fe60e7da)

All remaining items closed. Branch: 16 commits on origin/main 702f5be1b2f (PR#499+#504+#501 under the stack after 4 rebases), check.py --backend dynasm ALL PASSED 177/177.

• Phase 0 + B1 (double-apply class) — landed and verified (Phase-0 decline, executed-nonpure latch, Stage-1-3 forward-commit/forward-delivery, GC-rooted stashes, entry-clear parity fixes, 10 synth guards). Unchanged by the rebases; all repros oracle-equal on the final base.
• JIT: inlined-closure freevar read + branch miscompile — freevar cell aliases the branch-arm return slot (wrong value / TypeError crash) #498 (Family B miscompile) — RESOLVED separately; see JIT: inlined-closure freevar read + branch miscompile — freevar cell aliases the branch-arm return slot (wrong value / TypeError crash) #498 (fix 4266d987f86: branch-guard snapshots prefer the live walk register over the stale vable-shadow local binding; +3 synth guards 466d1511aee).
• task#1 (ForIterNext exemption audit, codex-review §3) — CLOSED as latent, masked: 11 shapes swept, zero default-flag divergence; the exemption's double-advance is reachable only under PYRE_FBW_NESTED_RESID_ABORT=0 on the old base (deterministic +1), i.e. fbw_abort_nested_unjournaled_residual is load-bearing. Regression guards landed (e2f8b533401, foriter_exempt_shared_generator / foriter_exempt_nested_foriter); on the current base even the lever no longer reproduces (+1 closed by the PR#499/JIT: inlined-closure freevar read + branch miscompile — freevar cell aliases the branch-arm return slot (wrong value / TypeError crash) #498-era fixes).
• Phase 1 / Slice A (callee operand-stack mirror) — infra landed gated OFF (b34fe60e7da, PYRE_FBW_CALLEE_VSTACK, 177/177 both modes); default flip NOT landed, by measurement: a corpus-wide PYRE_FBW_DEBUG_ABORT=1 census (all 177 synth files + fib_recursive/inline_helper/nbody) found the target decline (BranchGuardUnrestorableKeptStackPermanent in an inline sub-walk) fires nowhere; the only 2 kept-stack declines in the corpus are top-level (subwalk=false, invalid mirror — iteration_protocol, seqiter_getitem_lazy) which a callee mirror cannot address (verified: gate ON changes nothing there). Per the perf-neutral rule, added complexity without a measurable beneficiary does not flip; the gate stays as re-evaluation infra.
• Side find: probing for Slice-A census targets surfaced a pre-existing FBW walker-capture panic (not this epic's machinery; epic kill-switches irrelevant) → filed as jit: FBW panic in collect_outer_active_boxes — liveness-active ref register holds NONE (3-term or-chain + fresh-alloc call arg) #511 with a 13-line repro.

— updated by Claude

[youknowone]

Resolved. The epic's FINAL STATUS is reached via PR #513: the Phase-0 floor, the B1 forward-commit chain, and exception forward-delivery all landed, with 13 regression guards under bench/synth/ (inline_subwalk_*, foriter_exempt_*, closure_freevar_branch_*). The Slice-A infra landed gated-off (no measured beneficiary). The side-find #511 was fixed separately.

— commented by Claude