Skip to content

Prepares openeo-udp 0.2.0 for publishing on PyPI - #88

Open
firzaariany wants to merge 7 commits into
mainfrom
feat/pypi-packaging
Open

firzaariany wants to merge 7 commits into
mainfrom
feat/pypi-packaging

Conversation

@firzaariany

@firzaariany firzaariany commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Summary

This PR closes #87 and #82

This PR gets the openeo_udp package ready to publish on PyPI. Notebooks will then be able to %pip install openeo-udp, including in APEx JupyterLite. The parameter files now ship inside the package. A local file with the same name still overrides the packaged one.

This PR does not change the notebooks. That work is in feat/notebooks-pypi-install, which is based on this branch and should be merged after it.

Changes

Parameter files ship with the package.

  • Moved 36 *.params.py files from notebooks/** into openeo_udp/params/ and declared them as package data. ParameterManager("<id>.params.py") looks for the file in this order:
    • A file with that name in the current directory.
    • The packaged copy
      A path that includes a directory never falls back to the packaged copy.
  • New list_parameter_files(), exported from openeo_udp.
  • print_options() now starts with the file it loaded, for example Parameter file: urban_areas.params.py (packaged with openeo-udp). It warns when a local file overrides a packaged one.
  • The README has a new "Using your own parameter file" section showing how to copy a packaged file and edit it.

Standalone ML notebooks stay out of the package

  • The parameter files for forest_fire_dNBR and burned_forest_classification_ml stay in their notebook folders.
    Each of those folders now has a requirements.txt for its heavier dependencies (geopandas, scikit-learn, xarray and others).

Packaging metadata

  • Version 0.2.0. pyproject.toml reads it from openeo_udp.__version__, so it is defined in one place.
  • Authors: Emmanuel Mathot and Firza Riany.
  • Runtime dependencies are cut down to openeo and ipywidgets. Plotting libraries move to a notebooks extra.
  • The tooling that isn't part of the package moves into dependency groups: dev, batch (pydantic) and docs.
    uv.lock is regenerated.

Docs
I updated the parameter file paths in README.md, CONTRIBUTING.md, LICENSING.md and docs/parameter-management.md.

Tests

The new tests/test_packaged_params.py checks that:

  • Every packaged parameter file loads from a directory with no local copy.
  • A local file overrides the packaged one, and a missing name raises FileNotFoundError.
  • A path that includes a directory doesn't fall back to the packaged copy.
  • No *.params.py files are left in notebooks/ outside the two ML folders, and those two folders' files are not packaged.
    describe_source() and print_options() report where the file came from.
uv sync
uv run pytest tests/test_packaged_params.py

Merge order

This PR is the first of two stacked PRs:

  • This PR (feat/pypi-packaging → main): packaging, the parameter files, and the CI workflows (tests, build check, publish).
  • feat/notebooks-pypi-install → feat/pypi-packaging: puts the pinned %pip install cell in every notebook, removes in-notebook registration, and limits the wheel to openeo_udp/. When this PR merges, retargets the second PR to main.

Please merge this PR with a merge commit or rebase-merge, not a squash. A squash leaves this PR's commits in the second PR, so its branch would need a git rebase --onto before it can merge cleanly.

No release until both PRs are merged. Between the two merges, main has a wheel that still contains algorithm_registration, and notebooks that still call register(). That intermediate state is safe because:

  • The publish workflow runs only on v* tags, so merging it doesn't publish anything.
  • The pypi environment requires a reviewer's approval, so even an accidental tag can't publish on its own.

Release, after both PRs are merged:

  • Tag v0.2.0rc1. This publishes to TestPyPI, where we check the install in APEx JupyterLite.
  • Tag v0.2.0. This publishes to PyPI.

Notes for reviewers

  • The CI test workflow and the publish workflow (Trusted Publishing, with a TestPyPI dry run for rc tags) will come in a follow-up PR on this branch.
  • Poll: Suggested package name

firzaariany and others added 7 commits November 12, 2026 12:06
Moves all 38 *.params.py files from notebooks/ into openeo_udp/params/ so
they ship with the package on PyPI. Notebooks keep calling
ParameterManager("<algorithm>.params.py"): a bare file name that does not
exist locally now resolves to the packaged copy, while a local file with
the same name still takes precedence.

- Adds list_parameter_files() to list the packaged parameter files
- Replaces the notebook package-data rule with openeo_udp/params/*.params.py
- Adds tests that every packaged file loads, the local override and missing files
- Updates LICENSING.md, CONTRIBUTING.md, docs and the dNBR README to the new paths

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The forest_fire_dNBR and burned_forest_classification_ml notebooks are
self-contained and should not ship with openeo-udp, so their parameter
files move back next to their notebooks. They still load through
ParameterManager because a local file takes precedence over the
packaged lookup.

- Adds STANDALONE_NOTEBOOK_DIRS to the packaging tests as the allowed
  exceptions, and checks their parameter files are not packaged
- Restores the forest_fire_dNBR README to its original layout

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Bumps the version to 0.2.0, read by pyproject.toml from
  openeo_udp.__version__ so the two cannot drift apart
- Adds Firza Riany to the authors
- Reduces the core dependencies to openeo and ipywidgets, the only
  third-party packages the package imports
- Adds a "notebooks" extra (matplotlib, numpy, Pillow, rioxarray) for the
  packaged notebooks; the dev group installs it, so uv sync is unchanged
  for contributors
- Moves jupyter and ipykernel to the dev group and drops pip and requests
- Moves the ML-only packages (geopandas, scikit-learn, scipy, netcdf4,
  rasterio, xarray, shapely) into requirements.txt files in the
  forest_fire_dNBR and burned_forest_classification_ml folders; drops
  h5netcdf since netcdf4 covers reading the .nc results
- Adds a "batch" dependency group with pydantic for scripts/run_batch_job.py

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- print_options() now starts with the loaded parameter file and whether
  it is packaged or local, and warns when a local file overrides a
  packaged one of the same name
- Adds ParameterManager.is_packaged and describe_source(), with tests
- README explains that parameter files ship in openeo_udp/params/ and
  how to override one with a local copy

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
fapar, lava_visualization, ndci_cyanobacteria and urban_areas had no schemas;
flood_mapping and multitemporal_burnt_area lacked them on their runtime
parameters.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@firzaariany

Copy link
Copy Markdown
Contributor Author

I've cleaned up the dependencies. This took some time because I had to evaluate which dependencies were used by a handful of notebooks (e.g. fores_fire_dNBR and burned_forest_classification_ml), add new functions to parameter_manager.py to allow local parameter files to override packaged parameters, and experiment ways to let users specify their own.

What's left is to separate the doc for the package (PACKAGE.md) from that of the repo (README.md), and publish the package to PyPI. I've already prepared its integration in #89.

We also want to rename the package into something more descriptive. I was thinking of openeo-portable, as it reflects the package's role in managing backend connection of openeo processes. I'm not very inspired to come up with other names, so if the name sounds good to you, I'd suggest we go ahead with it @olafveerman @emmanuelmathot

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Publish package on PyPi?

1 participant