jit: expose builtin gateways and preserve recursion budget

[youknowone]

What changed

• install len through an interp2app-style wrapper and expose its translated JitCode
• make the existing isinstance wrapper expose the positional argument-read shape used by builtin gateway descent
• size the native byte budget for the worst-case JIT guard-failure resume chain
• run Python-created threads with a 20 MiB stack, yielding a 15 MiB effective guard budget after the existing one-quarter safety margin
• add and correctly classify a parity regression covering recursion depth before and after JIT warmup

Root cause and final stack sizing

The original macOS test_deep_nested_struct_frozenset regression was reproducible only under load and raised RecursionError. The old native guard reserved 2.8 MiB for a recursion limit of 1000. A Python recursion level that enters compiled code and resumes through a failed guard costs about 6.3 KiB of native stack, so the native guard could fire far before the Python recursion limit.

The first fix used a 64 MiB default worker stack. That removed the pickle failure, but it over-reserved address space per worker and let CPython's support.infinite_recursion(20_000) JSON test spend longer than its 300-second timeout below the native guard.

The final sizing keeps the 48 MiB requested budget used for JIT resume accounting, but gives ordinary runtime threads a 20 MiB native stack. The existing OS clamp leaves a 15 MiB effective guard: enough for the measured hot-resume cost at the default limit and for the 2000-level cold/hot parity fixture, while still bounding intentionally huge recursion limits. It changes no public recursion-limit value and adds no baseline FAIL/SKIP.

Validation

• re-extracted pyre-interpreter and pyre-jit LLBC; prepass completed with the existing 416 known Charon warnings
• cargo fmt --all -- --check
• cargo check --features dynasm
• cargo test --features dynasm
• all parity tests: 92/92 passed on both CPython and dynasm
• CPython gate: 209 PASS, 0 FAIL, 0 CRASH, 0 TIMEOUT
• targeted gates: test_struct 2/2, test_pickle 3/3, test_json 1/1
• concurrent full test_pickle.py: 12/12 passed at concurrency 6
• recursion cold/JIT-hot/thread fixture: passed
• full non-synthetic benchmark/selfcheck gate: dynasm 17/17 passed

[coderabbitai]

Warning

Review limit reached

@youknowone, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 25 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: a6ee839d-cec4-4595-b2ee-769a97734db9

📥 Commits

Reviewing files that changed from the base of the PR and between f5e308b and b5f45fa.

📒 Files selected for processing (4)

• pyre/extra_tests/parity_tests/recursion_limit_survives_jit_warmup.py
• pyre/pyre-interpreter/src/builtins.rs
• pyre/pyre-interpreter/src/stack_check.rs
• pyre/pyrex/src/lib.rs

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

🤖 Codex parity review

Static analysis of this diff vs the local RPython/PyPy sources (commit b5f45fa).
Updated: 2026-08-12T22:27:22.590Z

Files in the reviewed diff

pyre/extra_tests/parity_tests/recursion_limit_survives_jit_warmup.py
pyre/pyre-interpreter/src/builtins.rs
pyre/pyre-interpreter/src/stack_check.rs
pyre/pyrex/src/lib.rs

1. Regressions to PyPy parity introduced by this patch

• pyre/pyrex/src/lib.rs:434 ↔ rpython/translator/c/src/stack.c:50-53 — startup now installs a synthetic 20 MiB OS-stack size for the interpreter thread, so the clamp uses 15 MiB; PyPy derives its clamp solely from process RLIMIT_STACK. This regresses the prior main-thread fallback, which matched the upstream lookup.

2. Other mismatches introduced by this patch

None.

3. Pre-existing mismatches (already present before this patch)

None.

4. Structural adaptations

• pyre/pyre-interpreter/src/stack_check.rs:89 ↔ rpython/translator/c/src/stack.h:13-18 — 48 << 18 replaces PyPy’s architecture-selected 768 KiB/2.8 MiB byte budget. This is a Rust-frame/JIT-resume-stack sizing adaptation, not a direct RPython constant port.

• pyre/pyre-interpreter/src/stack_check.rs:118 ↔ pypy/module/sys/vm.py:73-76 — Pyre supplies a 20 MiB default native stack for Python-created Rust threads; PyPy delegates non-main-thread stack sizing to threading.stack_size()/the platform.

• pyre/pyre-interpreter/src/builtins.rs:4486-4501 ↔ pypy/module/__builtin__/operation.py:36-38 — the manual len gateway wrapper and descriptor have no PyPy-level analogue; they expose a source-translatable builtin-code graph while preserving space.len(w_obj) semantics.

• pyre/pyre-interpreter/src/builtins.rs:5934-5940 ↔ pypy/module/__builtin__/abstractinst.py:233-238 — the updated isinstance wrapper explicitly reads positional slots before invoking baseobjspace::isinstance; this is translator/JIT heap-key shape plumbing, with the same two-argument abstract-instance operation.

• pyre/extra_tests/parity_tests/recursion_limit_survives_jit_warmup.py:1-83 ↔ pypy/module/sys/vm.py:64-97 — the new hot-JIT recursion-stability test is Pyre-specific coverage for Rust compiled-resume native-stack cost; PyPy’s source has no corresponding cold-versus-hot depth test.

[chatgpt-codex-connector]

💡 Codex Review

https://github.com/youknowone/pyre/blob/3af0d52b3a5a143c1b397faab3b9ef15dd0dc7db/pyre-interpreter/src/stack_check.rs#L115
Avoid reserving 64 MiB for every default worker

When _thread.stack_size() remains zero, every Python-created worker now reserves 64 MiB instead of 8 MiB. In processes with an address-space limit or many threads, these reservations can exhaust virtual memory and make otherwise lightweight workers fail with can't start new thread—for example, a 1 GiB address-space limit leaves room for fewer than 16 such stacks before accounting for the rest of the process. The surrounding comments identify the larger reservation as compensation for the JIT guard-resume chain's amplified native-stack use, so that root cause should be fixed or the recursion budget decoupled rather than multiplying every worker's reservation.

AGENTS.md reference: AGENTS.md:L252-L254

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".