Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: Adopt pip-compile to improve requirements handling #194

Open
tmc opened this issue Nov 28, 2023 · 0 comments
Open

proposal: Adopt pip-compile to improve requirements handling #194

tmc opened this issue Nov 28, 2023 · 0 comments

Comments

@tmc
Copy link

tmc commented Nov 28, 2023

I propose adopting pip-tools as a way to more flexibly and reliably manage the contents in requirements.txt.

Current Situation

Right now the requirements are managed in a bit of a hard to use manner and this is likely a hinderance to folks attempting to work on native Mac support (See #154 and #184).

The codebase today does not have a mechanism to define and or test on different platforms, effectively leaving it CUDA-only. There is also not a straightforward way to define tests and requirements against different python or CUDA versions.

The dependencies are managed between the ./requirements/p2.txt (I have a proposed simplification at #193) and along the path to running at test as defined in https://github.com/Stability-AI/generative-models/blob/main/pyproject.toml#L45

Rationale
Only specifying one pip-freeze produced requirements file has a number of downsides:

  • It's often a post-hoc description of some system being in some working state, instead of being a source-controlled definition of requirements.
  • Adopting pip-compile means that version resolution becomes a development-time step instead of a build or installation time step.
  • pip-compile grants us an easier to manage and more reproducible description format while enabling an easy path to targeting multi-platform support.

Possible Downsides and Tradeoffs

  • pip-compile is another tool folks have to learn or cope with, this might mitigated by documentation and scripting/tooling to assist.

Alternatives Considered

  • Current approach.
  • Adopting poetry.
@tmc tmc changed the title proposal: Adopt pip-compile to manage requirements proposal: Adopt pip-compile to improve requirements handling Nov 29, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant