Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sync CryoCloud environment #18

Merged
merged 4 commits into from
Aug 7, 2024
Merged

Sync CryoCloud environment #18

merged 4 commits into from
Aug 7, 2024

Conversation

scottyhq
Copy link
Member

@scottyhq scottyhq commented Aug 7, 2024

Addresses #17 (comment)

Since CryoCloud has everything in our existing environment + a bunch of other stuff, the easiest thing seems to be just to overwrite the current environment.yml. However, I commented a bunch of stuff that is user-interface specific and also things that are not on mac-osx since I like having the ability to work on different platforms

Comment on lines +26 to +33
# https://syncthing.net/ provides a dropbox-like interface for syncing directories across users,
# hubs and machines. Used along with `jupyter-syncthing-proxy`.
# - jupyter-syncthing-proxy==1.0.3
# - syncthing~=1.22.1
# gh-scoped-creds allows users to securely push to GitHub from their repo.
# https://blog.jupyter.org/securely-pushing-to-github-from-a-jupyterhub-3ee42dfdc54f has
# some more information
# - gh-scoped-creds==4.1
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could not lock the environment for platform osx-arm64
Could not solve for environment specs
The following packages are incompatible
├─ code-server ~=4.16.1  does not exist (perhaps a typo or a missing channel);
├─ jupyter-syncthing-proxy 1.0.3  is not installable because it requires
│  └─ syncthing, which does not exist (perhaps a missing channel);
└─ syncthing ~=1.22.1  does not exist (perhaps a typo or a missing channel).

Comment on lines +146 to +162
# Desktop tools whose versions are more recent on conda-forge than ubuntu
#- qgis~=3.38.0
# Maybe needed for qgis? https://github.com/conda-forge/qgis-feedstock/issues/263
#- pyopencl
# Resolves warning "No ICDs were found": https://github.com/CryoInTheCloud/hub-image/issues/50
#- ocl-icd-system
# needed for Linux desktop to work
#- websockify>=0.10
# Includes new subcommands (`switch` & `restore`) and vulnerability fixes
# over the version available via ubuntu sources
#- git>=2.39

# Not all packages will be available from conda-forge, we install from pip when we need to.
#- pip~=23.3.2
#- pip:
# Access linux desktop from inside JupyterHub
# - jupyter-desktop-server==0.1.3
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are some big packages here (qgis) that don't seem necessary for executing tutorial notebooks, but can bring them back if needed...

@scottyhq scottyhq added the preview Create a website preview label Aug 7, 2024
@scottyhq scottyhq requested review from jomey and weiji14 August 7, 2024 00:04
@weiji14
Copy link
Member

weiji14 commented Aug 7, 2024

There's this error on running Sphinx build:

Running Sphinx v5.1.1

Extension error:
Could not import extension build_gallery (exception: cannot import name 'ExtensionMetadata' from 'sphinx.util.typing' (unknown location))
Traceback (most recent call last):
  File "/home/runner/micromamba/envs/hackweek/lib/python3.11/site-packages/sphinx/registry.py", line 442, in load_extension
    mod = import_module(extname)
          ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/micromamba/envs/hackweek/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/runner/work/website-2024/website-2024/book/_ext/build_gallery.py", line 12, in <module>
    from sphinx.util.typing import ExtensionMetadata
ImportError: cannot import name 'ExtensionMetadata' from 'sphinx.util.typing' (unknown location)

Looking to see which dependency is preventing a newer version of Sphinx from being installed...

Edit: It looks like sphinx.util.typing.ExtensionMetadata was made public in Sphinx 7.3.0 (sphinx-doc/sphinx#12153), so we'll either need to update to that version, or remove that ExtensionMetadata type hint from the 'book/_ext/build_gallery.py' file.

conda/environment.yml Outdated Show resolved Hide resolved
Copy link
Member

@weiji14 weiji14 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple of suggestions, should hopefully fix the ImportError: cannot import name 'ExtensionMetadata' from 'sphinx.util.typing' error. I'll update these over at https://github.com/CryoInTheCloud/hub-image too in a bit.

conda/environment.yml Outdated Show resolved Hide resolved
conda/environment.yml Outdated Show resolved Hide resolved
Copy link
Member

@jomey jomey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I on board with this approach. Commented on a few lines for a little more aggressive reduction of the library stack to ease up resolving the environment to a working state.

conda/environment.yml Outdated Show resolved Hide resolved
Copy link
Contributor

github-actions bot commented Aug 7, 2024

@scottyhq
Copy link
Member Author

scottyhq commented Aug 7, 2024

Thank you @weiji14 & @jomey for the helpful input here. Going to merge this!

@scottyhq scottyhq merged commit 5236364 into main Aug 7, 2024
4 checks passed
@scottyhq scottyhq deleted the cryocloud-sync branch August 7, 2024 19:13
weiji14 added a commit to CryoInTheCloud/hub-image that referenced this pull request Aug 7, 2024
Bumps [awscli](https://github.com/aws/aws-cli) from 1.27.76 to 2.17.2.
- [Release notes](https://github.com/aws/aws-cli/releases)
- [Commits](aws/aws-cli@1.27.76...2.17.2)

Also need to downgrade ipython from 8.20 to 8.17 to resolve conflict on prompt_toolkit version range (xref ICESAT-2HackWeek/website-2024#18 (comment)).
scottyhq added a commit to CryoInTheCloud/hub-image that referenced this pull request Aug 14, 2024
Syncing with the changes at
ICESAT-2HackWeek/website-2024#18 by:

- Pinning `sphinx` explicity to 7.3 (was resolving to 5.1.1 before)
- Upgrade `awscli` from 1.27.76 to 2.17 to resolve conflict on
`docutils` (see
ICESAT-2HackWeek/website-2024#18 (comment))
- Downgrade `ipython` from 8.20.0 to 8.17 to resolve conflict on
`prompt_toolkit` (see
ICESAT-2HackWeek/website-2024#18 (comment))

This PR also adds a 'conda-lock.yml' (following #122) in addition to the
existing 'conda-lock-linux64.yml' file.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
preview Create a website preview
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants