Shrink Ray gets stuck in replace_bodies_with_ellipsis at beginning of reduction

[devdanzin]

The attached sample file makes Shrink Ray get stuck for hours in replace_bodies_with_ellipsis for me. When this happens, it pegs one CPU core and memory use seems to remain constant.

The command used to run Shrink Ray is:

shrinkray --parallelism 4 --timeout=120 _lsprof_abort.sh source_lsprof.py

Where _lsprof_abort.sh is:

#!/bin/bash
/home/ubuntu/projects/upstream_cpython/python ./source_lsprof.py
if ! test "$?" = "134"; then
    exit 1
else
    exit 0
fi

Running c-reduce works on this file, and the result of such reduction works fine in Shrink Ray.

IIUC one feature of Shrink Ray is choosing suitable passes so the start isn't too slow, so this looks like a bug.

Example output:

                                                                      Shrink Ray. Press q to exit.
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Current test case size: 664044 bytes
Total runtime: 3 hours, 28 minutes and 26.15 seconds
Calls to interestingness test: 380 (0.03 calls / second, 0.00% interesting, 0.00% wasted)
No reductions yet
Reducer status: Running reduction pass replace_bodies_with_ellipsis
Current parallel workers: 4 (Average 2.23) (effective parallelism: 2.23)
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

python -VV:

Python 3.12.7+ (heads/3.12:512a5bd, Nov 20 2024, 21:38:00) [GCC 13.2.0]

Thank you very much for your wonderful tools and writings!
source_lsprof.py.txt

[DRMacIver]

this looks like a bug

Yeah, this is definitely a bug. Shrinkray really shouldn't get stuck in long-running passes, particularly early on when other passes would work. Will try to investigate soon.

Is the upstream cpython you're testing against some particular commit?

[devdanzin]

Is the upstream cpython you're testing against some particular commit?

Great question, I'll report the commit soon and see what happens in release tags.

Sorry for not checking what happens in different versions.

[devdanzin]

Sorry for the long delay, I mistakenly thought it was not reproducing and spent a lot of time reconstructing the conditions.

Turns out it was reproducing, just that my initial diagnosis was wrong: it doesn't get stuck at the first pass, but rather goes through some passes without any reduction.

For me, it went through: delete_statements, replace_statements_with_pass, hollow, split(b'\n')/delete_duplicates, split(b'\n')/block_deletion(10, 100), lift_braces, remove_indents, remove_whitespace, plus a couple more in between I didn't catch.

upstream_cpython is:

Python 3.12.8+ (heads/3.12:b0615a8a9aa, Dec 16 2024, 16:01:26) [GCC 13.3.0]

It only repros with 3.12 (where the CPython bug exists).