Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Installing a package editably in one envs given diamond hierarchy forces all cases to be editable #413

Open
PeterJCLaw opened this issue Aug 1, 2023 · 4 comments

Comments

@PeterJCLaw
Copy link
Contributor

In a monorepo setup it's useful to be able to install base packages as editable (-e) in development, though in production you want to be able to install them fully into the site-packages so that you don't need to cart around the source tree separately from the python environment.

It looks like this mostly works for a basic environment hierarchy, however something isn't quite right when there's a diamond hierarchy involved:

      ┌──→ left ──↘
root ─┼──→ right ─→ dev
      └→ base-dev ↗

In the setup I have, a local package is referenced (non-editable) in root and thus transitively by both left and right. base-dev installs it editable. dev just pulls everything together for convenience.

I realise this is a slightly complicated setup, so I've put a reproduce at https://github.com/PeterJCLaw/pip-compile-multi-diamond-hierarchy-bug-demo.

In the failing case, all references to the common utils package are forced to be editable.

Removing either left.in or right.in from dev.in causes the result I want -- only things which depend on base-dev.in actually get forced to have the local package installed editable.

@PeterJCLaw PeterJCLaw changed the title Support installing packages editably in some envs given diamond hierarchy Installing packages editably in some envs given diamond hierarchy forces all cases to be editable Aug 1, 2023
@PeterJCLaw PeterJCLaw changed the title Installing packages editably in some envs given diamond hierarchy forces all cases to be editable Installing a package editably in one envs given diamond hierarchy forces all cases to be editable Aug 1, 2023
@peterdemin
Copy link
Owner

JFYI: I acknowledge the issue but I don't have bandwidth to fix it. You're welcome to propose a solution and I'll review the PR

@PeterJCLaw
Copy link
Contributor Author

That's ok. Do you have any suggestions on why this might be working in the ordinary diamond hierarchy? (i.e: the case with only two, not three, legs)

@PeterJCLaw
Copy link
Contributor Author

Hrm, removing --autoresolve also yields the result I'm after.

@PeterJCLaw
Copy link
Contributor Author

Figured out why removing left.in or right.in from dev.in works -- because then the autoresolve logic can't find a common "sink" file which references all the other environments.

I think I now also understand why this is happening -- because when we do have a common sink and are using --autoresolve, the auto-resolving happens by setting a compatibilities constraints file for pip-compile which includes all the packages from the dev file, which is the editable version of the package.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants