Can't create environment locally

[r-ford]

I'm trying to run the notebook locally in order to review it for the gallery, and when I try to create the environment (with mamba on Ubuntu 22.04), I get the following error:

Could not solve for environment specs
Encountered problems while solving:
  - package planetary-computer-0.2.2-pyhd8ed1ab_0 requires pydantic >=1.7.3, but none of the providers can be installed

The environment can't be solved, aborting the operation

@pritamd47, have you run into this problem before?

[brian-rose]

Hmmm I just ran

mamba env create -f environment.yml

and successfully created the environment on a Mac laptop.

[brian-rose]

I tried removing the pyviz channel from the environment file, consistent with our not-yet-published review guidelines which favor conda-forge wherever possible.

I also successfully created the environment this way. It resulted in a different environment solution. In particular, conda-forge provided version 1.0.0 of planetary-computer rather than version 0.2.2. I suspect that this will resolve the issue on linux. @r-ford can you verify this?

@pritamd47 is there a specific reason that the pyviz channel is needed here?

[ktyle]

Yes, I was able to do the same just now on our department's Linux system. I notice that the version of planetary-computer now available is 1.0.0 (it was 0.2.2 in @r-ford 's attempt) so perhaps that fixed things.

Otherwise, I wonder if the pydantic<2 pin is necessary, and the same for the pyviz channel.

I see Brian is thinking on the same lines ... :)

[ktyle]

The environment builds fine with no pydantic < 2 pin (current version is 2.1.1). It wasn't clear in my post above, but I had also removed the pyviz channel specification. It seems that the various packages that have traditionally been made available on that channel are regularly updated on conda-forge these days.

Next I'll run through the notebooks to verify they run as expected.

[brian-rose]

This is all pointing to the importance of two bullets on our (still unpublished) draft review criteria:

• Install everything from conda-forge whenever possible
• Avoid pinning package version except where needed

And in both cases add comments to the environment file to explain why the exceptions are needed.

[r-ford]

I was able to create the environment by removing the pyviz channel and unpinning pydantic, but now I'm getting this error when trying to start up jupyter lab:

$ jupyter lab

Traceback (most recent call last):
  File "/home/rf194656/mambaforge/envs/interactive-sentinel-2-cookbook-dev/bin/jupyter-lab", line 6, in <module>
    from jupyterlab.labapp import main
  File "/home/rf194656/mambaforge/envs/interactive-sentinel-2-cookbook-dev/lib/python3.11/site-packages/jupyterlab/__init__.py", line 7, in <module>
    from .handlers.announcements import (  # noqa
  File "/home/rf194656/mambaforge/envs/interactive-sentinel-2-cookbook-dev/lib/python3.11/site-packages/jupyterlab/handlers/announcements.py", line 14, in <module>
    from jupyter_server.base.handlers import APIHandler
  File "/home/rf194656/mambaforge/envs/interactive-sentinel-2-cookbook-dev/lib/python3.11/site-packages/jupyter_server/base/handlers.py", line 23, in <module>
    from jupyter_events import EventLogger
  File "/home/rf194656/mambaforge/envs/interactive-sentinel-2-cookbook-dev/lib/python3.11/site-packages/jupyter_events/__init__.py", line 3, in <module>
    from .logger import EVENTS_METADATA_VERSION, EventLogger
  File "/home/rf194656/mambaforge/envs/interactive-sentinel-2-cookbook-dev/lib/python3.11/site-packages/jupyter_events/logger.py", line 18, in <module>
    from .schema import SchemaType
  File "/home/rf194656/mambaforge/envs/interactive-sentinel-2-cookbook-dev/lib/python3.11/site-packages/jupyter_events/schema.py", line 7, in <module>
    from referencing import Registry
ModuleNotFoundError: No module named 'referencing'

I've never heard of this referencing module, so I tried adding it to the environment and got this:

Could not solve for environment specs
Encountered problems while solving:
  - package jupyter-book-0.10.1-pyhd8ed1ab_0 requires jupytext >=1.8.0,<1.11, but none of the providers can be installed

Seems like the problem might be on my end.

[pritamd47]

Hi all, thanks for looking into this! After following the discussion, I tried recreating the environment in a linux machine (Ubuntu 22.04) after removing pyviz and unpinning pydantic, but I am also facing the same issue now as @r-ford - getting the ModuleNotFoundError: No module named 'referencing' while starting jupyter lab.
I also tried creating the environment using the environment.yml file in its current state (i.e., including pyviz channel and pinning pydantic to <2), but weirdly, I am getting the same error. I am not sure what is causing this as I do not remember facing this issue earlier in the work laptop where I developed the cookbook - which I don't have access to any more unfortunately. I am trying to see if I can modify the environment.yml file to fix this issue.

[pritamd47]

I pushed some changes to the environment file (#3) which seems to fix these issues on the linux machine I've been using to test. I would appreciate if anyone else can confirm if the changes fix the issues at their end.

[pritamd47]

Update: I see that the automatic build failed because I also removed jupyter-book from the environment file. However, adding it back to the environment file gives me the same - ModuleNotFoundError: No module named 'referencing' - error again. Here is what conda list | grep jupyter looks like:

jupyter-book              0.15.1             pyhd8ed1ab_0    conda-forge
jupyter-cache             0.6.1              pyhd8ed1ab_0    conda-forge
jupyter-lsp               2.2.0              pyhd8ed1ab_0    conda-forge
jupyter_bokeh             2.0.4              pyhd8ed1ab_0    conda-forge
jupyter_client            8.3.0              pyhd8ed1ab_0    conda-forge
jupyter_core              5.3.1           py311h38be061_0    conda-forge
jupyter_events            0.7.0              pyhd8ed1ab_0    conda-forge
jupyter_server            2.7.0              pyhd8ed1ab_0    conda-forge
jupyter_server_terminals  0.4.4              pyhd8ed1ab_1    conda-forge
jupyterlab                4.0.4              pyhd8ed1ab_0    conda-forge
jupyterlab_pygments       0.2.2              pyhd8ed1ab_0    conda-forge
jupyterlab_server         2.24.0             pyhd8ed1ab_0    conda-forge
jupyterlab_widgets        3.0.8              pyhd8ed1ab_0    conda-forge
sphinx-jupyterbook-latex  0.5.2              pyhd8ed1ab_0    conda-forge

[ktyle]

This looks to be related to jupyter-book/jupyter-book#2044 .

[ktyle]

The problem looks to be a conflict in dependencies between those of jupyterlab and jupyter-book ... in particular, how they are installed via conda-forge.

jupyterlab has as one of its dependencies the jupyter_events package.
The current version of jupyter_events, 0.7.0, now includes in its feedstock recipe a dependency on the references package.
references requires that attrs package be at >= 23.0

Meanwhile, jupyter-book has as one of its dependencies the sphinx-external-toc package.
sphinx-external-doc's conda recipe appears to require that attrs be < 22.0.

As a result of the this incompatibility between attrs versions, jupyter-book cannot be installed.

If one clones the current sphinx-external-toc repo, it appears that the need for attrs was eliminated during a code refactor.

Therefore it seems to me the problem may ultimately stem from sphinx-external-toc's feedstock recipe.

Pinning jupyter_events<0.7 will result in the environment successfully building. However this would be a short-term fix, and would likely need to be put in https://github.com/ProjectPythia/cookbook-template/blob/main/environment.yml since I don't think any new cookbooks will build properly without it.

The longer-term fix would be to look into the sphinx-external-toc condaforge recipe.