-
-
Notifications
You must be signed in to change notification settings - Fork 29.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Py_GIL_DISABLED not defined in "free-threaded" Windows C-API extensions #111650
Comments
@zooba, I'd appreciate your feedback and suggestions about the best way to make sure that Windows C API extensions have |
I was curious about how GHA passed the test, but I noticed that it was skipped from the GHA..
|
I think we probably need to start modifying pyconfig.h at some stage in the build process. Eventually we should get an improved sysconfig [alternative] that can include the flag, but until then I don't think we want to make modifications in every build backend that's out there, nor do we want to force them to launch the target interpreter in order to get the right settings (which prevents cross-compilation, for example). I'm not sure how best to do it though. Probably the ideal looks like:
We shouldn't have to patch anything in the stdlib that finds the header, I'm pretty sure they were only in distutils. And an install layout should have it end up in the So there's a lot of moving parts, plus we need to figure out the best way to template the header so it can be modified as part of the build. There's not really anything built into our build system for this yet. |
Hi @zooba, I started implementing this and ran into a few issues. Not having a |
Prior to this change, the Py_NOGIL macro was not defined when building C API extensions with the `--disable-gil` build on Windows. `Py_NOGIL` was only defined as a pre-processor definition in pyproject.props, but that is not used by C-API extensions. This instead generates the `pyconfig.h` header on Windows as part of the build process. For now, `Py_NOGIL` is the only macro that may be conditionally defined in the generated file.
Prior to this change, the Py_NOGIL macro was not defined when building C API extensions with the `--disable-gil` build on Windows. `Py_NOGIL` was only defined as a pre-processor definition in pyproject.props, but that is not used by C-API extensions. This instead generates the `pyconfig.h` header on Windows as part of the build process. For now, `Py_NOGIL` is the only macro that may be conditionally defined in the generated file.
Prior to this change, the Py_NOGIL macro was not defined when building C API extensions with the `--disable-gil` build on Windows. `Py_NOGIL` was only defined as a pre-processor definition in pyproject.props, but that is not used by C-API extensions. This instead generates the `pyconfig.h` header on Windows as part of the build process. For now, `Py_NOGIL` is the only macro that may be conditionally defined in the generated file.
Prior to this change, the Py_NOGIL macro was not defined when building C API extensions with the `--disable-gil` build on Windows. `Py_NOGIL` was only defined as a pre-processor definition in pyproject.props, but that is not used by C-API extensions. This instead generates the `pyconfig.h` header on Windows as part of the build process. For now, `Py_NOGIL` is the only macro that may be conditionally defined in the generated file.
Left a few suggestions on the draft PR - in short:
|
POSIX places the generated
If we don't move
Sure - I'll convert the script to Python. |
If we don't move it, then a whole lot of other stuff needs updating instead. But we should 100% move it, because we don't put build-generated files into the source tree - they always live in the build tree, which does not overlap. (POSIX doesn't work like this, but people who have thought about it say that it should've, and that writing build files into the source tree is an unfortunate design choice. Our Windows build gets this right, and I don't want to regress it.) |
I don't know how to implement what you are asking for and I've already spent a few days on this, so I'm going to defer this and come back to it later. |
I'll get it figured out. Chances are I'm going to have to do all the integration into the release process anyway, so I may as well go end-to-end. |
Is there anything left to do here? |
Bug report
On Windows, the
Py_GIL_DISABLED
macro is not defined when building C API extensions with the free-threaded build of CPython 3.13 (i.e.,--disable-gil
). Now that biased reference counting (6dfb8fe) landed, this leads to incorrect/failed compilation of extensions on Windows 1. The problem does not occur on Linux/macOS and other./configure
based builds because that build system defines thePy_NOGIL
macro as part ofpyconfig.h
. On Windows,Py_GIL_DISABLED
instead defined as a pre-processor definition inpyproject.props
, but that's used by C-API extensions.Among other problems, this leads to
test_peg_generator
failing on the nogil Windows buildbot (example failure).I can think of two possible approaches to fixing this:
Py_GIL_DISABLED
when invoking the compiler ifsysconfig.get_config_var("Py_NOGIL")
isTrue
.Py_GIL_DISABLED
depending on if the build is invoked with--disable-gil
or not.The first option seems a little more straightforward to me, but there is a risk that we might encounter this same issue across multiple extension build systems. The second option seems like it would more robustly address this issue.
Linked PRs
Footnotes
Modules built as part of the CPython build (i.e., standard library modules) still work ↩
The text was updated successfully, but these errors were encountered: