Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

gh-111650: Ensure pyconfig.h includes Py_GIL_DISABLED on Windows #112778

Merged
merged 10 commits into from
Dec 13, 2023
Merged

gh-111650: Ensure pyconfig.h includes Py_GIL_DISABLED on Windows #112778

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
4feb2ce
gh-111650: Ensure pyconfig.h includes PY_GIL_DISABLED on Windows
zooba Dec 5, 2023
0243c40
Fix includes for nogil and tests
zooba Dec 6, 2023
094ccd9
Merge remote-tracking branch 'upstream/main' into gh-111650
zooba Dec 6, 2023
85811ea
Merge remote-tracking branch 'upstream/main' into gh-111650
zooba Dec 7, 2023
c33aa01
Fix include path for updated pyconfig.h
zooba Dec 7, 2023
d0b30bb
Expanded output when building
zooba Dec 7, 2023
c80c852
Move old pyconfig, fix updates
zooba Dec 7, 2023
313618b
Fix test
zooba Dec 8, 2023
6c067b7
Remove additional guards
zooba Dec 11, 2023
c74133b
Add extra include dir for PEG tests
zooba Dec 12, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Lib/sysconfig/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -404,7 +404,7 @@ def get_config_h_filename():
"""Return the path of pyconfig.h."""
if _PYTHON_BUILD:
if os.name == "nt":
inc_dir = os.path.join(_PROJECT_BASE, "PC")
inc_dir = sys.prefix
else:
inc_dir = _PROJECT_BASE
else:
Expand Down
6 changes: 5 additions & 1 deletion Lib/test/test_sysconfig.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -472,11 +472,15 @@ def test_srcdir(self):
# should be a full source checkout.
Python_h = os.path.join(srcdir, 'Include', 'Python.h')
self.assertTrue(os.path.exists(Python_h), Python_h)
# <srcdir>/PC/pyconfig.h always exists even if unused on POSIX.
# <srcdir>/PC/pyconfig.h always exists even if unused
pyconfig_h = os.path.join(srcdir, 'PC', 'pyconfig.h')
self.assertTrue(os.path.exists(pyconfig_h), pyconfig_h)
pyconfig_h_in = os.path.join(srcdir, 'pyconfig.h.in')
self.assertTrue(os.path.exists(pyconfig_h_in), pyconfig_h_in)
if os.name == 'nt':
# <prefix>/pyconfig.h exists on Windows in a build dir
pyconfig_h = os.path.join(sys.prefix, 'pyconfig.h')
self.assertTrue(os.path.exists(pyconfig_h), pyconfig_h)
elif os.name == 'posix':
makefile_dir = os.path.dirname(sysconfig.get_makefile_filename())
# Issue #19340: srcdir has been realpath'ed already
Expand Down
3 changes: 3 additions & 0 deletions Misc/NEWS.d/next/Windows/2023-12-05-22-56-30.gh-issue-111650.xlWmvM.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Ensures the ``Py_GIL_DISABLED`` preprocessor variable is defined in
:file:`pyconfig.h` so that extension modules written in C are able to use
it.
8 changes: 5 additions & 3 deletions PC/layout/main.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,10 @@ def copy_if_modified(src, dest):
)

if do_copy:
shutil.copy2(src, dest)
try:
shutil.copy2(src, dest)
except FileNotFoundError:
raise FileNotFoundError(src) from None


def get_lib_layout(ns):
Expand Down Expand Up @@ -208,8 +211,7 @@ def _c(d):

for dest, src in rglob(ns.source / "Include", "**/*.h"):
yield "include/{}".format(dest), src
src = ns.source / "PC" / "pyconfig.h"
yield "include/pyconfig.h", src
yield "include/pyconfig.h", ns.build / "pyconfig.h"

for dest, src in get_tcltk_lib(ns):
yield dest, src
Expand Down
3 changes: 3 additions & 0 deletions PC/pyconfig.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -739,4 +739,7 @@ Py_NO_ENABLE_SHARED to find out. Also support MS_NO_COREDLL for b/w compat */
/* Define if libssl has X509_VERIFY_PARAM_set1_host and related function */
#define HAVE_X509_VERIFY_PARAM_SET1_HOST 1

/* Define if you want to disable the GIL */
#undef Py_GIL_DISABLED
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you want to rename the PC/pyconfig.h? It seems like some tools might treat the input template as if it's the true pyconfig.h file.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not actually a template, it's a totally valid and usable header file. In the default case (for now), Py_GIL_DISABLED is undefined, and we just change that on build if requested.

For editing in an IDE, we need some kind of actual header file available or else all the code assistance tools will fail. Rather than requiring the IDE understand the build directory (which VS can do, but others may not), this allows us to have a default GIL_DISABLED header before building and then the correct one when actually building.


#endif /* !Py_CONFIG_H */
3 changes: 1 addition & 2 deletions PCbuild/pyproject.props
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,9 +38,8 @@
</PropertyGroup>
<ItemDefinitionGroup>
<ClCompile>
<AdditionalIncludeDirectories>$(PySourcePath)Include;$(PySourcePath)Include\internal;$(PySourcePath)Include\internal\mimalloc;$(PySourcePath)PC;$(IntDir);%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
<AdditionalIncludeDirectories>$(PySourcePath)Include;$(PySourcePath)Include\internal;$(PySourcePath)Include\internal\mimalloc;$(IntDir);$(PySourcePath)PC;%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
<PreprocessorDefinitions>WIN32;$(_Py3NamePreprocessorDefinition);$(_PlatformPreprocessorDefinition)$(_DebugPreprocessorDefinition)$(_PydPreprocessorDefinition)%(PreprocessorDefinitions)</PreprocessorDefinitions>
<PreprocessorDefinitions Condition="'$(DisableGil)' == 'true'">Py_GIL_DISABLED=1;%(PreprocessorDefinitions)</PreprocessorDefinitions>
zooba marked this conversation as resolved.
Show resolved Hide resolved
<PreprocessorDefinitions Condition="'$(SupportPGO)' and ($(Configuration) == 'PGInstrument' or $(Configuration) == 'PGUpdate')">_Py_USING_PGO=1;%(PreprocessorDefinitions)</PreprocessorDefinitions>

<Optimization>MaxSpeed</Optimization>
Expand Down
25 changes: 25 additions & 0 deletions PCbuild/pythoncore.vcxproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -644,6 +644,31 @@
<Import Project="regen.targets" />
</ImportGroup>
<Target Name="_TriggerRegen" BeforeTargets="PrepareForBuild" DependsOnTargets="Regen" />

<Target Name="_UpdatePyconfig" BeforeTargets="PrepareForBuild">
<MakeDir Directories="$(IntDir)" Condition="!Exists($(IntDir))" />
<ItemGroup>
<PyConfigH Remove="@(PyConfigH)" />
<PyConfigH Include="@(ClInclude)" Condition="'%(Filename)%(Extension)' == 'pyconfig.h'" />
</ItemGroup>
<Error Text="Did not find pyconfig.h" Condition="@(ClInclude) == ''" />
<PropertyGroup>
<PyConfigH>@(PyConfigH->'%(FullPath)', ';')</PyConfigH>
<PyConfigHText>$([System.IO.File]::ReadAllText($(PyConfigH)))</PyConfigHText>
<OldPyConfigH Condition="Exists('$(IntDir)pyconfig.h')">$([System.IO.File]::ReadAllText('$(IntDir)pyconfig.h'))</OldPyConfigH>
</PropertyGroup>
<PropertyGroup Condition="$(DisableGil) == 'true'">
<PyConfigHText>$(PyConfigHText.Replace('#undef Py_GIL_DISABLED', '#define Py_GIL_DISABLED 1'))</PyConfigHText>
</PropertyGroup>
<WriteLinesToFile File="$(IntDir)pyconfig.h" Lines="$(PyConfigHText)" Condition="$(PyConfigHText) != $(OldPyConfigH)" />
</Target>
<Target Name="_CopyPyconfig" Inputs="$(IntDir)pyconfig.h" Outputs="$(OutDir)pyconfig.h" AfterTargets="Build" DependsOnTargets="_UpdatePyconfig">
<Copy SourceFiles="$(IntDir)pyconfig.h" DestinationFolder="$(OutDir)" />
</Target>
<Target Name="_CleanPyconfig" AfterTargets="Clean">
<Delete Files="$(IntDir)pyconfig.h;$(OutDir)pyconfig.h" />
</Target>

<Target Name="_GetBuildInfo" BeforeTargets="PrepareForBuild">
<PropertyGroup>
<GIT Condition="$(GIT) == ''">git</GIT>
Expand Down
2 changes: 1 addition & 1 deletion Tools/msi/dev/dev_files.wxs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
<Fragment>
<ComponentGroup Id="dev_pyconfig">
<Component Id="include_pyconfig.h" Directory="include" Guid="*">
<File Id="include_pyconfig.h" Name="pyconfig.h" Source="!(bindpath.src)PC\pyconfig.h" KeyPath="yes" />
<File Id="include_pyconfig.h" Name="pyconfig.h" Source="pyconfig.h" KeyPath="yes" />
</Component>
</ComponentGroup>
</Fragment>
Expand Down
Loading