What if the Jupyter Launcher had a "Python 3 (hatch)" option that gave notebooks the environment they need?

[itcarroll]

What do you think about writing a kernel provisioner to manage notebook environments? The goal would be to let hatch manage environments for notebooks like hatch run can manage environments for scripts. Given the right metadata, of course, which isn't standardized yet. Chicken, meet egg. This has never been done before.

In Jupyter, kernel provisioners provide a new way for third-parties to manage the environment in which the kernel process runs.

In jupyter_client, I've just opened an issue regarding the storage of metadata in a notebook for use by the provisioner. I'd like to find a third-party who could use that metadata, and I think hatch does a fantastic job supporting big ideas that benefit users.

[ofek]

I don't have much time to look into this; could you please explain a bit more how this would be used?

[itcarroll]

Yes, I can explain however much you need (and that I understand)!

The hatch run command, when pointed at a python script, creates a virtual environment and installs the packages listed in the inline script metadata. We have nothing like that for Jupyter notebooks; it's a longstanding issue.

A notebook author (or a tool) would add dependencies to their notebook. Then, a Jupyter kernel that used the requested hatch kernel provisioner, would not use the local python environment. It would use an environment managed by hatch, that is equipped with packages to run the notebook.

[ofek]

That helps, thanks. Where would this provisioner package live and how would it be installed? Trying to wrap my head around how all the pieces fit in.

[itcarroll]

I believe the provisioner would be a subclass of jupyter_client.provisioning.LocalProvisioner and could be included in hatch itself along with an entry point into jupyter_client.kernel_provisioners. It would be used when any kernel shown in jupyter kernelspec list was suitably configured (using the metadata.kernel_provisioner stanza). That kernelspec would be the one for which I've suggested the display name "Python 3 (hatch)". It would still use ipykernel as the actual kernel. I'm not sure yet when or who would install this kernelspec.