Question: Is There a Way to Enable Extras in tool.scikit-build.overrides?

[ahnaf-tahmid-chowdhury]

Context

I am working on a project where I want to enable conditional features, specifically MPI support, using Python extras during installation:

pip install .[mpi]

I have defined optional dependencies in my pyproject.toml like this:

[project.optional-dependencies]
mpi = ["mpi4py"]

I want to conditionally enable MPI in the build configuration, and I tried using the [[tool.scikit-build.overrides]] section to detect when the mpi extra is used:

[[tool.scikit-build.overrides]]
if = "extra == 'mpi'"
cmake.define.ENABLE_MPI = "ON"

However, this results in an error:

"extra == 'mpi'" is not valid under any of the given schemas.

Question

Is there currently a way to conditionally apply overrides based on extras (e.g., extra == 'mpi') within the tool.scikit-build.overrides section?

Suggestion (If Not Currently Supported)

If this feature is not currently supported, would it be possible to introduce support for matching extras within the overrides section? This would make it easier to enable specific build configurations based on user-specified extras, aligning with Python’s optional-dependencies mechanism.

Thank you for your time and assistance!

Environment Details

• scikit-build-core version: 0.10.7
• Python version: 3.12
• Platform: Linux

[LecrisUT]

Unfortunately that would be incompatible with PyPI packaging where including an optional-dependency does not add, change, or remove files being installed, only dependencies installed change.

What you could do is support packaging ecosystems that do support such a split like conda, spack and distro packaging, but that is particular to each of those systems. As long as you can control it via a tool.scikit-build.cmake.define you should be able to design a form compatible with all those frameworks (a lot of careful designing for the distro case though). See the guide for how to control tool.scikit-build options within the pip install cli

[henryiii]

The way people sometimes do this is they make a second package with the MPI components, then include that via an extra. So pkg[mpi] would include pkg-mpi, where you put all the mpi components. You then can design it so that pkg will look for pkg_mpi's components and use them if present.

Or, of course, you can have a custom define, and make different "pkg" wheels, but keep in mind only one wheel with the same name can be uploaded to PyPI.

If you can always compile with MPI, but only activate it when mpi4py is present, that could work too. It would make building the wheel a bit harder (but you could provide the cmake.define option for binary builds hosted on PyPI, and not require it for src builds).

(But, yes, PyPI packaging doesn't allow extras to modify a wheel like Rust does).

[ahnaf-tahmid-chowdhury]

Is there an example available that uses MPI while creating a wheel? From what I understand about working with MPI, it needs to be installed manually on the user's side at first. Then, when a user installs the MPI version of the package, it starts building the package during the installation through the wheel. I am not sure how scikit-build-core handles this.