Skip to content

properly normalize / validate abiflags and include in environment signature - #3299

Merged
messense merged 5 commits into
PyO3:mainfrom
davidhewitt:abiflags
Sep 11, 2026
Merged

messense merged 5 commits into
PyO3:mainfrom
davidhewitt:abiflags

Conversation

@davidhewitt

Copy link
Copy Markdown
Member

maturin currently generates PYO3_ENVIRONMENT_SIGNATURE without accounting for abiflags. This in particular means that builds switching between regular and free-threaded Python can fail to invalidate pyo3-ffi's cache and lead to compilation selecting the wrong ABI.

This PR corrects the generated environment signature, and attempts to make abiflag handling correct throughout maturin. In particular I found the following other deficiencies:

  • Parsing PYO3_CONFIG_FILE did not infer (or validate) missing ABI flags from Py_GIL_DISABLED or Py_DEBUG.
  • The default CPython extension suffix generated from PYO3_CONFIG_FILE omitted free-threaded and debug flags. (This would produce a wheel with an invalid extension for the free-threaded/debug interpreter.)
  • Cross-compilation sysconfig parsing did not infer missing ABI flags from build settings.
  • Interpreter discovery generally did not account for the possibility of the d debug abiflag.

Comment thread src/python_interpreter/abiflags.rs Outdated

@messense messense left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we also update here?

print("version_major", build_time_vars["VERSION"][0]) # 3
print("version_minor", build_time_vars["VERSION"][2:]) # E.g., 8, 10
KEYS = [
"ABIFLAGS",
"EXT_SUFFIX",
"SOABI",
"Py_GIL_DISABLED",
]
for key in KEYS:
print(key, build_time_vars.get(key, ""))

@davidhewitt

Copy link
Copy Markdown
Member Author

Thanks, addressed both points. To keep the ordering consistent I simplified to either generate abiflags matching packaging if no flags present, otherwise I just validate the user-supplied flags at least match the sysconfigdata keys.

@davidhewitt

Copy link
Copy Markdown
Member Author

conda bug looks possibly unrelated?

@messense
messense merged commit bce7e5e into PyO3:main Sep 11, 2026
42 of 45 checks passed
kylebarron added a commit to kylebarron/arro3 that referenced this pull request Oct 12, 2026
All entries in the test-python matrix shared one Swatinem/rust-cache key,
so they restored the same `target/` directory. maturin (1.14 and 1.15)
sets the same PYO3_ENVIRONMENT_SIGNATURE for 3.14 and 3.14t, and
PYO3_PYTHON is `.venv/bin/python` in both jobs. So when 3.14t had saved
the cache, pyo3's build script didn't rerun in the 3.14 job, and arro3
was built against the free-threaded config. Importing it failed with
`undefined symbol: PyUnstable_Module_SetGIL`.

Upstream maturin fixed the signature in PyO3/maturin#3299, which isn't
released yet. A per-version cache key avoids the problem regardless.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants