Building Python 3.13 dev releases + no GIL

[jakirkham]

Python 3.13 has 2 alpha releases and a 3rd one due out in a week. Here's the release schedule for more details

Of particular note, the --disable-gil build flag was added, which should allow building a Python with the GIL disabled. It's important to note that this mode is not ABI compatible with the mode that keeps the GIL. So any extension module built with Python would need to be rebuilt with the no GIL interpreter to be used with it. That said, in no GIL mode there is an environment variable to toggle GIL usage, PYTHONGIL. Eventually the ABI stability will be addressed, but this is still several Python releases away. So we will need to decide what to do before then

Naively would propose building both build modes

We may also want to explore building a couple common libraries (like NumPy, SciPy, scikit-learn) to try with. IIRC some libraries already have fixes included for no GIL. So those seem like a preferable starting point

There is some interest in building out packages with Anaconda. So think we will want to coordinate

Would be interested to hear thoughts on this 🙂

[ovo-Tim]

Hi, I'm trying to build NoGil. I met some problems while building for Linux on github action.
Can you have a look? https://stackoverflow.com/questions/77895008/build-cpython-failed-on-github-action-output-tons-of-error-messages

Also, I have built successfully on my own computer. See: https://anaconda.org/ovo-tim/nogil

[xhochy]

@ovo-Tim Can you make a pull request to the feedstock with your changes? Then we could integrate that here (and help you with CI)

[ovo-Tim]

@xhochy Should I create a new branch, or just pull up github action?

[xhochy]

Please make a pull request with a branch on your repository and keep using Azure Pipelines. We don't want to use GitHub Actions for package builds.

[ovo-Tim]

@xhochy Hi, you may should create a branch. Than I wil make a pull request with this new branch. (I think I don't have permission to create a branch on this repository, I am not a git expert.)

[xhochy]

No, please fork and make a pull request from that fork. GitHub has good documentation on how to contribute via a pull request to another repository. Please have a look at that.

[ovo-Tim]

Should I pull to main branch?

[h-vetinari]

See also conda-forge/conda-forge.github.io#1990

[h-vetinari]

Adding to the excitement (disruption?) of 3.13, it now has a build-time dependence on LLVM (haven't checked yet which bits).

[jakirkham]

Indeed in typical Python fashion there is a poem to go with it: python/cpython#113465

[h-vetinari]

Haha, nice. :)

@h-vetinari: [...] build-time dependence on LLVM (haven't checked yet which bits).

Needs clang 16 and llvm-readobj, details here. Cross-compilation should work, "since the JIT is built for the host platform" - though I don't see how that would then imbue the target architecture with a working JIT... 🤔

[xhochy]

though I don't see how that would then imbue the target architecture with a working JIT... 🤔

The copy-and-patch JIT actually generates Assembly at compile-time and only "patches" that assembly together at runtime. There is no compilation happening at runtime.

[h-vetinari]

Thanks for the explanation @xhochy! But wouldn't those assembly pieces have to be specific to the target architecture? I guess that would work if the JIT on host can target another architecture - in that case I just got confused by "JIT built only on host" not clearly implying that.

[xhochy]

I think it might be confusing vocablulary. I would assume host == target are the same.

[jakirkham]

There is now an environment variable/flag to configure whether the GIL is enabled in Pythons built with --disable-gil

xref: python/cpython#116338