CMake fails to find SWIG on rebuilds #869
Comments
|
You must turn off build isolation ( |
|
Oh, wait, this is when you are not using editable installs with automatic rebuilds? We are supposed to automatically update the stored paths in the cache to the new isolated environment. Will investigate tomorrow. |
|
This is #420, and I think we don't handle this correctly yet. I'll bump it up in priority to investigate. We were supposed to process the cache and replace these but it looks like we haven't implemented that yet, I'd like to try that before trying to bundle and unbundle the build environment. |
|
If the build directory is reused it kinda makes sense. You can try to either add |
|
I think #420 could still hit the issue because I don't quite see how the hash is pulling in the build isolation dependencies. How about checking if there are any other dependencies in the |
|
Thanks for the recommendations and information! It sounds like the current work arounds are as follows:
I tested using the last option, and it successfully rebuilt without issue. It looks like it even cached the C++ library and only recompiled the SWIG interface, which is nice. |
That's CMake for ya ;). I was looking just now to see how to add the |
|
I think we have the info to do this, and we can start with |
Problem
Thanks for such an awesome package. I have really enjoyed learning how to use scikit-build-core.
When installing a SWIG wrapped library with scikit-build-core, rebuilds fail when I have the
build-dirset to a static location. This occurs whether installing in editable mode or not.Details
I am using SWIG to build a Python package for a C++ library. To assist with initial development and debugging, I have set
tool.sckit-build.build-dir = "build/{cache_tag}"to point a static build folder. This works great and successfully builds on a "fresh"pip install. However, when rebuilding (without completely deleting the build folder) it fails during the CMake configuration step. Specifically, it fails because it cannot find SWIG:Cmake Error
After some searching, I was able to figure out that the CMakeCache.txt in the build directory is not updating the temporary, isolated environment in which SWIG is installed to by pip. The weird part, is that Ninja is properly updated in the cache file. So it seems that CMake is picking up the new location for NInja, but using the old cached location for SWIG?
Fresh Build
For example, here is the directory
pipinstalls Ninja and SWIG to on a fresh build. This successfully builds:Installing collected packages: swig, tomli, pathspec, packaging, exceptiongroup, scikit-build-core Creating /tmp/pip-build-env-ykto9qfq/overlay/bin changing mode of /tmp/pip-build-env-ykto9qfq/overlay/bin/swig to 755 changing mode of /tmp/pip-build-env-ykto9qfq/overlay/bin/swig4.0 to 755 ... Installing collected packages: ninja Creating /tmp/pip-build-env-ykto9qfq/normal/bin changing mode of /tmp/pip-build-env-ykto9qfq/normal/bin/ninja to 755And the cached locations in the corresponding CMakeCache.txt file:
Rebuild
Now, when rebuilding without deleting the build directory the pip install directories are updated:
Installing collected packages: swig, tomli, pathspec, packaging, exceptiongroup, scikit-build-core Creating /tmp/pip-build-env-xgzud74l/overlay/bin changing mode of /tmp/pip-build-env-xgzud74l/overlay/bin/swig to 755 changing mode of /tmp/pip-build-env-xgzud74l/overlay/bin/swig4.0 to 755 ... Installing collected packages: ninja Creating /tmp/pip-build-env-xgzud74l/normal/bin changing mode of /tmp/pip-build-env-xgzud74l/normal/bin/ninja to 755These are new temp install directories which is correct, but the CMakeCache.txt that is created/updated only reflects the new directory for Ninja and not for SWIG:
The SWIG path is the same as the fresh build. All other paths referencing
/tmp/pip-build-env-*are properly updated.I'm not exactly sure yet if it's a CMake, SWIG, pip, or scikit-build-core issue, but I figured I should start here because it seems to me it's somewhere between pip installing build system packages and invoking CMake. This can be avoided by either removing the
build-dirconfiguration, or by building in editable mode as described in the docs. However, it is useful to do cached builds without using editable installs.I'm happy to keep digging for the true culprit, but I wasn't sure the next best place to look.
Supporting Files
Here are the full outputs and cache files for a fresh install and rebuild:
Fresh Install: Command Log
Fresh Install: CMakeCache.txt
Rebuild Install: Command Log
Rebuild Install: CMakeCache.txt
Minimal Reproducible Example
pip install -v .twiceThe text was updated successfully, but these errors were encountered: