Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[pypi] Support requirements.txt lockfiles without extras in them #2399

Open
3 tasks
aignas opened this issue Nov 13, 2024 · 0 comments
Open
3 tasks

[pypi] Support requirements.txt lockfiles without extras in them #2399

aignas opened this issue Nov 13, 2024 · 0 comments

Comments

@aignas
Copy link
Collaborator

aignas commented Nov 13, 2024

As of now pip.parse has a requirement on being provided with a lockfile with the extras annotations in the file. This is not the default setting for pip-compile >=8.0 and uv pip compile. It also means that we are sometimes pulling unnecessary dependencies into the dependency closure. The way we could in theory remove this requirement is by changing the following parts.

  • In whl_library_targets macro generate py_library targets for each extra that the package provides. This could be done by asking the wheel_installer to return deps for each extra instead of only one when an extra argument is specified - this behaviour is better to be only enabled in bzlmod, but that is just a guiding note.
  • In the pip.parse, fetch the METADATA from PEP-658 compliant servers or by downloading the first whl in the selected distribution list and extracting the metadata in other cases.
  • Parsing the METADATA in the extension context to get the list of all of the extras that the wheel provides and passing that to the pip_hub repository_rule as the extra_hub_aliases field so that we can create extra aliases.

Lock files which have this information that we would need to retrieve from METADATA could be alternatively supported without fetching of the METADATA from the PyPI server (e.g. uv.lock has all of the info we need).

I am writing this idea down in case someone would like to contribute as I would be happy to mentor the contributor and help with finishing the design here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant