add 'no-pip-install' recipe

[RobertRosca]

Slight abomination of a 'package'.

When installed it adds an EXTERNALLY-MANAGED (PEP 668) file to the python stdlib directory, preventing direct pip install commands into the environment.

Behaviour is:

(tmp) [roscar@arch-desktop custom-recipes]$ mamba install -y no-pip-install -c ./conda-bld

...

(tmp) [roscar@arch-desktop custom-recipes]$ pip install numpy

error: externally-managed-environment

× This environment is externally managed
╰─> This is a pure conda environment, packages should not be installed into it directly via pip.
    
    If you are a user see this page for more information on using our environments: https://european-xfel.github.io/environments/environments/
    
    For DA staff see the environment maintenance documentation:
    
    - Environment maintenance: https://european-xfel.github.io/environments/maintenance/environments/
    - Recipe creation: https://european-xfel.github.io/environments/maintenance/recipes/

(tmp) [roscar@arch-desktop custom-recipes]$ mamba install numpy

# works as expected

It'll prevent (well, warn, since you can force the install anyway) us from pip installing things into the environment accidentally, and could help with the occasional... odd... packages which internally run pip install to 'help' simplify their use/setup.

My idea is to include it in the main cycle environments, for everything else it's up to whoever sets it up to decide.

[takluyver]

Does this also affect users loading our environment and doing pip --user installs to add an extra package? I know that comes with its own issues, but IMO it's sometimes a reasonable compromise for users (versus creating a whole new environment and possibly a Jupyter kernel).

[RobertRosca]

It would, yes, since the PEP was meant to help stop issues with system python installations. I remember reading the discussions around this and in the end the reasoning was that users can pip install something which goes into .local, but if there's some incompatibility between the packages installed by the system and packages set up in the local user environment it could cause issues with the user system, since applications can pick up that incompatible package.

I think any 'fix' for this would involve patching pip in some way, to skip the check if --user is passed, and/or to check for a different file. The relevant functions are:

• check_externally_managed https://github.com/pypa/pip/blob/7aaca9f2c43ba20b3efffe7fb1b9eb463677ccd7/src/pip/_internal/utils/misc.py#L645
• InstallCommand.run https://github.com/pypa/pip/blob/7aaca9f2c43ba20b3efffe7fb1b9eb463677ccd7/src/pip/_internal/commands/install.py#L275

Skipping would mean modifying InstallCommand.run 's if to include not options.use_user_site before running check_externally_managed, which would disable the check if --user is passed to the install command.

But then if somebody does python3 -m pip install --upgrade pip --user they'd get a non-patched version which would no longer let them install things 🙃

So in addition to that check_externally_managed should also look for a different file, like EXTERNALLY-MANAGED-CONDA, that way pip install would fail when in our environment, pass when used with --user, and pass when used with a newer version of pip already in ~/.local.

Patching pip like this gives me a feeling like... pulling hair out of a drain, but I can add a simple test case to check for this behaviour and we can pin no-pip-install to a single version of pip.

TBH, even though it's a small change, I'm not sure if this kind of patching ends up being worse than the risk of something accidentally being installed into the environment and breaking it 😟

[takluyver]

Yeah, I don't think we should go as far as patching pip; that's going to be a fair bit of work. I just wanted to be understand before we decide either way on using EXTERNALLY-MANAGED.

But it does mean I don't really know what I prefer. For preventing us from accidentally pip-installing into the environment, I think it's a straightforward win. For users, I tend to think that doing a user install of a package is probably better, even with the risk that it later breaks, than creating & managing their own environment (as this is something that a lot of our users aren't familiar with). But I don't know how many users it would affect. 🤷