WIP: Attempt to get pyyaml to build with cython 3.0 #607
Conversation
|
Yeah, just getting the libyaml extension to build under Cython 3.0 is pretty straightforward, as you've shown here- our usage of Cython is fairly basic. However, changing this would break every packager that's still calling setup.py directly and passing pyyaml-custom args to it, and wouldn't provide a dynamic fallback to pure-Python pyyaml for users that aren't installing a wheel but don't have a working extension build env. I'm not saying that's the wrong approach, since setuptools has long deprecated directly calling setup.py, but we'd need to provide other methods for packagers and users to accomplish the same configurations (probably via envvars, given the current state of PEP517 config data), and preferably a deprecation period for packagers to direct them to use the new methods (optional, but kind). Off the top of my head, the things we need to be able to support are:
I hacked envvar support for some of these into the existing setup.py code back when we added |
Fair, but in my experience with Matplotlib and h5py, down stream packagers adapt to even major changes in how to build a package so long as you provide clear examples of what they should do now.
I agree that is a pretty show-stopping problem, as are the first 3 bullet points. I think the cythonize mode + extension building should be grouped, getting cython is much easier than getting the needed complier and pre-cythonized c-code has a tendency to not be forward-compatible. A sdist that require cython is far more likely to be usable in the future than an sdist that has cythonized c-code and I am not convinced that the complexity of explaining / implementing how to force re-cythonization is worth the narrow case of someone who has the Python headers, libyaml, and a c-compiler but not cython (which can be pip installed!). |
Agreed- we quit including Cython'd outputs in our sdists back when we added pyproject.toml and made Cython a build dep for those exact reasons. We'd gotten nailed a few times having to quick-turn new sdist releases to work with breaking changes to the Python APIs generated by older versions of Cython while trying to keep things working under pre-release Pythons. |
|
I'd like to target some form of this work, probably along with the more dynamic discovery of libyaml @karolyi has been working on in #618 for the next non-patch PyYAML release. I've been hoping the PEP517 |
|
On the plus side, support for |
git clean -xdf
tar zcvf ../python-yaml_6.0.1.orig.tar.gz --exclude=.git .
debuild -uc -us
cp python-yaml.spec ../python-yaml_6.0.1-1.spec
mv ../python*-yaml*6.0.1*.{gz,xz,spec,dsc} /osc/home\:alvistack/yaml-pyyaml-6.0.1/
rm -rf ../python*-yaml*6.0.1*.*
See yaml#607
Signed-off-by: Wong Hoi Sing Edison <[email protected]>
This is cribbed from h5py's setup_build.py. While this works and the tests pass, it drops a lot of the configuration that used to be possible / support for other Pythons / setting the include path / ...
|
6.0.2 includes a "temporary" hack for Cython 3 extension build support |
|
Thank you, I can confirm the default branch builds cleanly with a dev version of cython. |
This is a slightly different approach to fixing #601 than #602 which drops the sub-classes and attempts to simplify the build logic.
This is cribbed from h5py's setup_build.py https://github.com/h5py/h5py/blob/master/setup_build.py
While this works and the tests pass I think it is dropping some configuration control and support for other Pythons.