v0.2 goals

[juntyr]

The cutoff for v0.2.0 will be 13.12.2024. The release will follow, at the latest on 19.12.2024.

• improve the CORS proxy: 205fec7 climet-eu/pyodide@03b4bea
  • only use proxy if necessary
  • warn on first usage of proxied address
  • perhaps move the proxy common code to a library
• implement persistent session folders
  • create a fresh session folder on startup
  • navigate to the fresh session folder on startup
  • copy the requirements.txt file to the sessions
    • Fix contents copy stripping part of path jupyterlite/jupyterlite#1524
  • upload git repos to the session directory
    • Upload to current working directory jupyterlite/litegitpuller#25
    • Upload to a user-provided uploadpath jupyterlite/litegitpuller#26
  • update the docs in this repo and the notebooks to explain the semi-persistence
• fix some content drive file size handling bugs
  • file size: Fix file size handling for the contents drive jupyterlite/jupyterlite#1530
  • large uploads: handle binary file chunks jupyterlite/jupyterlite#1497
  • Bump to jupyterlite deps to v0.4.5 jupyterlite/pyodide-kernel#146
• update all packages to the latest version, where possible
  • update packages
    • outdated, blocked by conflicts: botocore, cartopy, numcodecs, packaging, pandas, pyproj
    • outdated, blocked by patches: h5py, matplotlib, numpy
  • update fcbench
  • update to the latest upstream pyodide
  • Add values_dtype backend option to load values at full precision ecmwf/cfgrib#407
  • pyfdb should not depend on tox ecmwf/pyfdb#39
  • optional: https://gitlab.dkrz.de/m300602/pyicon

Bugs to fix:

• package loading log doesn't work, manual console.log is still captured
• lab sometimes runs out of memory quite quickly, possibly because of -s MAXIMUM_MEMORY=4GB
• earth earthkit-time package is missing
• pyarrow package is soft-missing
• aiohappyeyeballs package is missing, but also aiobotocore depends on aiohttp.http_exceptions being accessible, so this may be version incompatibility
• cdsapi has moved to the new datapi package
• extend the funding statement at the bottom of the intro notebook

[juntyr]

If pyodide/pyodide#4510 lands very soon, the following packages could still be updated

• matplotlib -> 3.8.4
• fcbench -> 0.0.14
• cartopy -> 0.24.1

[juntyr]

Upgrading to numpy v2 doesn't seem possible at the moment

• basemap still pins numpy < 2: Numpy 2.0 Compatibility  matplotlib/basemap#604
• pandas would need to be 2.2.2 at least
• covjsonkit (unsolicited dependency through earthkit -> earthkit-data[all]) pins pandas==2.2.0
  • Relax pandas version pin to support 2.2.x ecmwf/covjsonkit#56
• fcbench could be upgraded to support numpy v2

[agriyakhetarpal]

I would be happy to assist on the Pyodide side for any additional goals. :D

[juntyr]

I would be happy to assist on the Pyodide side for any additional goals. :D

@agriyakhetarpal Thank you for the offer, that's very kind!

I think especially once the building of packages moves out of the main pyodide repo, I'll be able to contribute a few extra packages from the climate and meteorology domain that I've been maintaining in this fork (the lab here is a combination of a pyodide fork with some extra packages and some extra, partially quite experimental patches for better JupyterLite integration).

When I will come across some issues with Pyodide or areas where I'd have questions, I'll reach out :)

[juntyr]

Published in https://github.com/climet-eu/lab/releases/tag/v0.2.0