Python 3.13 breaks circular imports during single phase init of extension module

[hauntsaninja]

Here's a bash script to reproduce:

printf '
#define PY_SSIZE_T_CLEAN
#include <Python.h>

static struct PyModuleDef nativemodule = {
    PyModuleDef_HEAD_INIT,
    .m_name = "native",
};

PyObject* module = NULL;

PyMODINIT_FUNC PyInit_native(void) {
    if (module) {
        Py_INCREF(module);
        return module;
    }
    module = PyModule_Create(&nativemodule);
    assert(module);

    Py_XDECREF(PyImport_ImportModule("non_native"));
    PySys_WriteStdout("hello from native\\n");

    return module;
}
' > native.c

printf 'import native  # circular import' > non_native.py

printf '
from setuptools import setup, Extension

setup(name="native", ext_modules=[Extension("native", sources=["native.c"])])
' > setup.py

python setup.py build_ext --inplace

python -c 'import native'

This produces:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
    import native
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 921, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 819, in module_from_spec
  File "<frozen importlib._bootstrap>", line 782, in _init_module_attrs
SystemError: extension module 'native' is already cached

and an assertion failure in debug builds.

The new error comes from #118532 cc @ericsnowcurrently
It's unclear whether this breaking change is intentional, given no mention in documentation and the assert.

This affects the mypyc transpiler, see python/mypy#17748 for details and for an end-to-end repro. This means for instance that mypy and black cannot currently be compiled for Python 3.13. Changing mypyc to use multi-phase init is not an easy change because mypyc uses globals.

CPython versions tested on:

3.13

Operating systems tested on:

No response

Linked PRs

• gh-123880: Allow recursive import of single-phase-init modules #123950