Skip to content

Add rusterize engine support for rasterization #61

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
dcherian merged 12 commits into from
Feb 4, 2026
Merged

Add rusterize engine support for rasterization #61

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
447d234
Add rusterize engine support for rasterization
dcherian Dec 9, 2025
d8260ce
fix test
dcherian Jan 23, 2026
11e1a26
Make geometry_clip engine-generic
dcherian Jan 24, 2026
8958a52
Fixes
dcherian Jan 24, 2026
c3f1a1c
Add snapshots
dcherian Jan 24, 2026
0f24632
Merge branch 'main' into rusterize
dcherian Feb 3, 2026
ce52b2a
Add error handling for invalid merge_alg in dask rasterize path
dcherian Feb 3, 2026
3434366
Add rusterize documentation page
dcherian Feb 3, 2026
c4edf0f
Update rusterize CI to test Python 3.14 instead of 3.13
dcherian Feb 3, 2026
2b8f220
Bundle NSIDC sea ice concentration TIFF for docs
dcherian Feb 3, 2026
7368d6e
Fix notebook imports for rasterize module refactor
dcherian Feb 3, 2026
9c1e752
Add rusterize to docs dependencies
dcherian Feb 3, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 46 additions & 0 deletions .github/workflows/test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,3 +68,49 @@ jobs:
# with:
# token: ${{ secrets.CODECOV_TOKEN }}
# verbose: true # optional (default = false)

test-rusterize:
name: rusterize, py=${{ matrix.python-version }}

strategy:
matrix:
python-version: ["3.11", "3.14"]
os: ["ubuntu-latest"]
runs-on: ${{ matrix.os }}

steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0 # grab all branches and tags
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
cache: "pip"
- name: Install Hatch
run: |
python -m pip install --upgrade pip
pip install hatch
- name: Restore cached hypothesis directory
id: restore-hypothesis-cache
uses: actions/cache/restore@v4
with:
path: .hypothesis/
key: cache-hypothesis-${{ runner.os }}-${{ github.run_id }}
restore-keys: |
cache-hypothesis-
- name: Set Up Hatch Env
run: |
hatch env create test-rusterize.py${{ matrix.python-version }}
hatch env run -e test-rusterize.py${{ matrix.python-version }} list-env
- name: Run Tests
run: |
hatch env run --env test-rusterize.py${{ matrix.python-version }} run-coverage

- name: Save cached hypothesis directory
id: save-hypothesis-cache
if: always()
uses: actions/cache/save@v4
with:
path: .hypothesis/
key: cache-hypothesis-${{ runner.os }}-${{ github.run_id }}
8 changes: 4 additions & 4 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,9 +52,9 @@ coverage.xml
# Sphinx documentation
docs/_build/
docs/api
docs/data
data
data.zip
docs/autoapi
/data
/data.zip

# PyBuilder
target/
Expand All @@ -80,7 +80,7 @@ src/rasterix/_version.py
#example*
#doesnotexist
#test_sync*
data/*
/data/*
src/fixture/
fixture/
junit.xml
Expand Down
Binary file added docs/data/S_20240101_concentration_v4.0.tif
View file
Open in desktop
Binary file not shown.
1 change: 1 addition & 0 deletions docs/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ caption: Rasterizing
hidden:
---
rasterize/rasterio
rasterize/rusterize
rasterize/geometry_mask
rasterize/exactextract
```
Expand Down
2 changes: 1 addition & 1 deletion docs/raster_index/crs.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@
},
"outputs": [],
"source": [
"source = \"/vsicurl/https://noaadata.apps.nsidc.org/NOAA/G02135/south/daily/geotiff/2024/01_Jan/S_20240101_concentration_v4.0.tif\""
"source = \"../data/S_20240101_concentration_v4.0.tif\""
]
},
{
Expand Down
2 changes: 1 addition & 1 deletion docs/raster_index/indexing.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@
},
"outputs": [],
"source": [
"source = \"https://noaadata.apps.nsidc.org/NOAA/G02135/south/daily/geotiff/2024/01_Jan/S_20240101_concentration_v4.0.tif\"\n",
"source = \"../data/S_20240101_concentration_v4.0.tif\"\n",
"\n",
"da = xr.open_dataarray(source, engine=\"rasterio\").pipe(rasterix.assign_index)\n",
"da"
Expand Down
2 changes: 1 addition & 1 deletion docs/raster_index/intro.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@
},
"outputs": [],
"source": [
"source = \"https://noaadata.apps.nsidc.org/NOAA/G02135/south/daily/geotiff/2024/01_Jan/S_20240101_concentration_v4.0.tif\""
"source = \"../data/S_20240101_concentration_v4.0.tif\""
]
},
{
Expand Down
4 changes: 2 additions & 2 deletions docs/rasterize/geometry_mask.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@
"metadata": {},
"outputs": [],
"source": [
"from rasterix.rasterize.rasterio import geometry_mask\n",
"from rasterix.rasterize import geometry_mask\n",
"\n",
"geometry_mask(ds, asia[[\"geometry\"]], xdim=\"longitude\", ydim=\"latitude\").plot()"
]
Expand All @@ -85,7 +85,7 @@
"metadata": {},
"outputs": [],
"source": [
"from rasterix.rasterize.rasterio import geometry_clip\n",
"from rasterix.rasterize import geometry_clip\n",
"\n",
"n = geometry_clip(ds, asia[[\"geometry\"]], xdim=\"longitude\", ydim=\"latitude\")\n",
"n.u.isel(month=1, level=0).plot()"
Expand Down
7 changes: 4 additions & 3 deletions docs/rasterize/rasterio.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,9 +64,9 @@
"metadata": {},
"outputs": [],
"source": [
"from rasterix.rasterize.rasterio import rasterize\n",
"from rasterix.rasterize import rasterize\n",
"\n",
"n = rasterize(ds, world[[\"geometry\"]], xdim=\"longitude\", ydim=\"latitude\")\n",
"n = rasterize(ds, world[[\"geometry\"]], xdim=\"longitude\", ydim=\"latitude\", engine=\"rasterio\")\n",
"n.plot()"
]
},
Expand All @@ -90,7 +90,7 @@
"outputs": [],
"source": [
"chunked = ds.chunk({\"latitude\": -1, \"longitude\": 120})\n",
"d = rasterize(chunked, world[[\"geometry\"]], xdim=\"longitude\", ydim=\"latitude\")\n",
"d = rasterize(chunked, world[[\"geometry\"]], xdim=\"longitude\", ydim=\"latitude\", engine=\"rasterio\")\n",
"d"
]
},
Expand Down Expand Up @@ -126,6 +126,7 @@
" dgpd.from_geopandas(world[[\"geometry\"]], npartitions=3),\n",
" xdim=\"longitude\",\n",
" ydim=\"latitude\",\n",
" engine=\"rasterio\",\n",
")\n",
"dd"
]
Expand Down
155 changes: 155 additions & 0 deletions docs/rasterize/rusterize.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,155 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# With rusterize\n",
"\n",
"Rasterix includes a dask-aware wrapper for [rusterize](https://github.com/ttrotto/rusterize), a fast Rust-based rasterization library.\n",
"\n",
"```{tip}\n",
"Unlike the `rasterio` engine, `rusterize` does **not** require GDAL! This makes it much easier to install and is a great choice when GDAL installation is problematic.\n",
"```\n",
"\n",
"```{note}\n",
"The `rusterize` engine does not support the `all_touched` parameter. Use `engine='rasterio'` if you need `all_touched` support.\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Read data\n",
"\n",
"### Read in some raster data"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import xarray as xr\n",
"import xproj # noqa\n",
"\n",
"ds = xr.tutorial.open_dataset(\"eraint_uvz\")\n",
"ds = ds.proj.assign_crs(spatial_ref=\"epsg:4326\")\n",
"ds"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Read in example geometries"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import geodatasets\n",
"import geopandas as gpd\n",
"\n",
"world = gpd.read_file(geodatasets.get_path(\"naturalearth land\"))\n",
"world"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from rasterix.rasterize import rasterize\n",
"\n",
"n = rasterize(ds, world[[\"geometry\"]], xdim=\"longitude\", ydim=\"latitude\", engine=\"rusterize\")\n",
"n.plot()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Out-of-core support\n",
"\n",
"All combinations of chunked and in-memory arrays and geometries are supported.\n",
"\n",
"### dask.array + geopandas"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"chunked = ds.chunk({\"latitude\": -1, \"longitude\": 120})\n",
"d = rasterize(chunked, world[[\"geometry\"]], xdim=\"longitude\", ydim=\"latitude\", engine=\"rusterize\")\n",
"d"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"xr.testing.assert_identical(n, d)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### dask.array + dask-geopandas"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import dask_geopandas as dgpd\n",
"\n",
"dd = rasterize(\n",
" ds.chunk({\"latitude\": -1, \"longitude\": 240}),\n",
" dgpd.from_geopandas(world[[\"geometry\"]], npartitions=3),\n",
" xdim=\"longitude\",\n",
" ydim=\"latitude\",\n",
" engine=\"rusterize\",\n",
")\n",
"dd"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"xr.testing.assert_identical(dd, d)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"language_info": {
"name": "python"
}
},
"nbformat": 4,
"nbformat_minor": 4
}
2 changes: 1 addition & 1 deletion notebooks/raster_index.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@
"metadata": {},
"outputs": [],
"source": [
"source = \"/vsicurl/https://noaadata.apps.nsidc.org/NOAA/G02135/south/daily/geotiff/2024/01_Jan/S_20240101_concentration_v4.0.tif\""
"source = \"../docs/data/S_20240101_concentration_v4.0.tif\""
]
},
{
Expand Down
Loading
Loading