Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support different backends for pipx? #1392

Open
huxuan opened this issue May 5, 2024 · 11 comments · May be fixed by #1487
Open

Support different backends for pipx? #1392

huxuan opened this issue May 5, 2024 · 11 comments · May be fixed by #1487
Labels
enhancement New feature or request

Comments

@huxuan
Copy link
Member

huxuan commented May 5, 2024

How would this feature be useful?

There are several open issues about supporting different "backends", such as #1370 for uv and #853 for virtualenv. Moreover, there is already a project pipxu that claims to have reimplemented pipx while using uv. So I would like to have your comments about whether we should support different backends.

Describe the solution you'd like

  1. Abstract all the operations related to managed virtual environments.
  2. Implement venv, virtualenv and uv for these abstracted operations.
  3. Add virtualenv and uv as optional dependencies and use them with higher priority than venv when pipx is initialized.

Describe alternatives you've considered

There might be two other approaches to implement the same feature:

  1. Create a backend plugin system and implement the backend in a separate repo.
  2. Create a standalone and universal backend library that can be used directly in pipx.
@dukecat0 dukecat0 added the enhancement New feature or request label May 5, 2024
@chrysle
Copy link
Contributor

chrysle commented May 5, 2024

  1. Abstract all the operations related to managed virtual environments.
  2. Implement venv, virtualenv and uv for these abstracted operations.

Sounds good, ans would be similar to what nox is already doing. We must prevent that uv steals our jobs ;-)!!

Actually, I find it really strange that someone just re-implements an actively developed program for a functionality that could be added to the same, and on his own, too!?

@cs01
Copy link
Member

cs01 commented May 6, 2024

the uv readme says:

Think: a single binary that bootstraps your Python installation and gives you everything you need to be productive with Python, bundling not only pip, pip-tools, and virtualenv, but also pipx, tox, poetry, pyenv, ruff, and more.

Our goal is to evolve uv into such a tool.

so if they are successful in implementing their roadmap, it will subsume all of these tools including pipx (and pipxu)

Ultimately it will be up to users if they want to use uv, but I do suspect users want a single well-built answer to Python packaging and common workflows. I think it’s a real possibility pipx becomes essentially deprecated in favor of uv. Not sure if or when that will happen exactly but I think it’s important to be realistic that this is a possible future.

It doesn’t mean pipx is bad - it means pipx was successful in making its feature set an expectation for Python packaging ecosystem. This is a win for Python users and the language itself.

Even if uv implements pipx, having competing tools is usually a good thing for users, since they push eachother to be better and faster. So +1 to having pipx use uv as a backend.

@huxuan
Copy link
Member Author

huxuan commented May 6, 2024

Hi @cs01, nice to meet you!

I agree with you completely, and personally I am eager to see that happen, like how 'ruff' replaced all lint tools. So, just as mentioned in the issue, do you have any comments whether we should think about making a single, universal backend library? Just like @chrysle said, 'nox' already implemented something similar. And I've seen 'pdm' made it too. We can avoid the reinventing of the wheels and advocate a consistent approach. This could be useful during migration.

@chrysle
Copy link
Contributor

chrysle commented May 6, 2024

So, just as mentioned in the issue, do you have any comments whether we should think about making a single, universal backend library?

Hmm, doesn't sound like a bad idea. However, for more projects than pipx to use it, it would have to be highly customizable. And there are different backends for different needs, e.g. installation, virtualenv creation, lockfile management etc., thus it would require differentiated modularization. And 'single, universal' sounds like a standard could be needed.

@huxuan
Copy link
Member Author

huxuan commented May 6, 2024

And 'single, universal' sounds like a standard could be needed.

Yep, that is the point, and I suppose PyPA might be the most suitable or even the only organization to do that.

@AdrianVollmer
Copy link

I'd love to see uv as a backend for pipx.

I think it’s a real possibility pipx becomes essentially deprecated in favor of uv. Not sure if or when that will happen exactly but I think it’s important to be realistic that this is a possible future.

Keep in mind that astral is VC backed, so there are investors who presumably want to see returns at some point. The interests of users and investors often don't exactly align. While I can see avenues that might be win-win, I can also see avenues resembling enshittifcation, so I think there will always be value in having and maintaining pipx.

@cs01
Copy link
Member

cs01 commented May 6, 2024

Nice to meet you too @huxuan :)

So, just as mentioned in the issue, do you have any comments whether we should think about making a single, universal backend library?

Would the backend library be in Rust, call into uv, and have python bindings for callers like pipx?

tbh I don't feel I am actively involved in the Python ecosystem or on pipx anymore to have a strong opinion, but it does sound pretty interesting. I'm just glad to see so much interest in continuing to push pipx and forward. Thanks so much.

@huxuan
Copy link
Member Author

huxuan commented May 7, 2024

Would the backend library be in Rust, call into uv, and have python bindings for callers like pipx?

What I mean is a Python wrapper for different modules with capability of managing virtual environments, such as venv, virtualenv, conda and uv. It should be almost the same as the abstraction and implementation pattern, but the advantange is that we can maintain a more robust library and use it in different projects.

@henryiii
Copy link
Contributor

I think there should be separation between --installer (uv or pip) and --venv-backend (venv, virtualenv, uv). I also like having a uv|pip option, which means use uv if available, fall back to pip.

The main questions, IMO, are:

  • Should uv|pip be the default, or should pip remain the default, for backward-compat? (build and nox went with the former, hatch is doing something inbetween, with all new features like PEP 723 using uv and older ones being opt-in).
  • Is there a way to make this configurable without having to pass a command line option? An envvar would make sense, but is there a place we could put a config option?

@huxuan
Copy link
Member Author

huxuan commented May 11, 2024

Hi @henryiii,

Thanks for the suggestions! It is quite informative. I did not realize that there should also have an --installer option.

For uv|pip option, a fallback strategy is definitely useful, but we may not need to speicify it explicitly. In my assumption, when we specify uv for the --installer and uv is not available, we can fallback to pip with a warning message. WDYT?

@pfmoore
Copy link
Member

pfmoore commented May 12, 2024

I would love to see uv as a backend for pipx. In particular, the speed of uv would make it ideal as an installer backend for pipx run, particularly when running a script. The run command is typically used for occasional tasks that don't necessarily benefit as much from caching the environment.

@dukecat0 dukecat0 linked a pull request Jul 22, 2024 that will close this issue
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants