Trying out Pixi for image building

[abkfenris]

I'm trying out using Pixi for managing our Conda and Pypi environment in place of Mamba.

pixi-kernel replaces nb-conda-kernels to allow us to support per-project environments.

I'm using features to capture each tutorial, then adding the tutorial feature to the default environment. The [dependencies] table captures our base dependencies, though we might want to refactor them out into features as well ('jupyterlab support', 'cli', 'storage', 'fast xarray',...).

You can test the lab locally with either pixi run lab or docker compose up py.

There is one little oddity I know of right now in the pixi workflow. If you create a new directory ((like a project), pixi init, then following pixi commands don't target that manifest as normal.

$ pixi init
Initialized project in /home/jovyan/test/.
$ pixi add fastapi
 WARN Using manifest /home/jovyan/pixi.toml from `PIXI_PROJECT_MANIFEST` rather than local /home/jovyan/test/pixi.toml

This is because the image has already set a pixi shell (well pixi shell-hook) to force pixi to look for a specific manifest and recorded that in an environment variable. Pixi will warn that it's using a different manifest, but its easy to screw that up.

Running pixi shell --manifest-path ./pixi.toml will set the pixi scope to the current environment.

$ pixi shell --manifest-path ./pixi.toml
 . "/tmp/pixi_env_WOz.sh"
jovyan@e5cb7d16120a:~/test$  . "/tmp/pixi_env_WOz.sh"
(test) jovyan@e5cb7d16120a:~/test$ pixi add fastapi

[abkfenris]

We might need/want to create our 'base' pixi environment in a different directory, so it doesn't get smushed by Jupyterhub mapping the user folder in place of /home/jovyan.

[abkfenris]

Added some packages to speed up Xarray based on pydata/xarray#9149

[abkfenris]

Landing this, then making PR for Callum's dependencies and sending to 2i2c so we can test it