Wheel package auto detection / editable install isuee

[tikuma-lsuhsc]

In #932, @henryiii mentioned that

Scikit-build-core installs your package directory automatically if it can find it (and even works in editable mode). Generally it's best to try to avoid installing .py files via CMake so you can fully take advantage of editable installs.

I cannot seem to get this feature to work. My project file structure looks like this:

myproject
├── pyproject.toml  # AND/OR setup.cfg, setup.py
├── CMakeLists.txt
├── ...
└── src/
    ├── CMakeLists.txt
    └── mypkg/
        ├── __init__.py
        ├── CMakeLists.txt
        └── core/
            ├── core_source.cpp
            ├── ...
            └── CMakeLists.txt

The root CMakeLists.txt includes the only install line:

install(TARGETS ${PROJECT_NAME} LIBRARY DESTINATION ./mypkg)

According to the documentation, src/mypkg should be automatically picked up as the wheel-install-dir and by @henryiii's comment above, __init__.py should be automatically included in the installation, After pip -e . install, I should be able to do import mypkg.core but gets ModuleNotFoundError. pip installed the .pyd file in site-packages/mypkg directory (no __init__.py) as well as _myproject_editable.* files in the site-packages directory.

[henryiii]

What's the package.name value? It needs to be mypkg for this to work. It looks for src/<name>, python/<name>, and <name>. Otherwise, you have to specify tool.scikit-build.wheel.packages. This matches the way hatchling works. Setuptools instead has a massive ignore list and looks for a package-like directory that is not in that ignore list, which does let you use differing names without configuration, but is fairly fragile.

[henryiii]

Personally, I'd move src/mypkg/core to a different spot, like src/core or core. As it is, you'll probably also want to exclude the src/mypkg/core directory from your wheel, as you don't need the source code installed.

[tikuma-lsuhsc]

What's the package.name value? It needs to be mypkg for this to work. It looks for src/, python/, and .

Ah, <name> is the package name and not any arbitrary name. I misunderstood it as the name of the first subdirectory. My package name is indeed different from distro name, so it sounds like I must use wheel.packages. (Reason: the module name has been already taken by another project's distro name in PyPI .)

It's likely too late to change the auto-detection behavior, but I'd prefer scikit-build-core to scan for the packages in src and define CMake variables for each top-level package found, e.g., SKBUILD_PACKAGE_mypkg. Too crazy of an idea?

Personally, I'd move src/mypkg/core to a different spot, like src/core or core. As it is, you'll probably also want to exclude the src/mypkg/core directory from your wheel, as you don't need the source code installed.

You're absolutely right. I didn't think of that side effect. This also explains why you suggest python/<name>!

[henryiii]

You should be really careful about mismatching names. It may come back to haunt you like pypa/packaging-problems#818. There's no general solution for an import name fight.

[tikuma-lsuhsc]

The mismatching-name concern, duly noted! Thank you for the clarification! I'm closing the issue now.