test notebooks in 'docs/', make cuspatial_api_examples self-contained, skip long-running notebook, fix some docs

[jameslamb]

Description

Contributes to #1406.

Unblocks CI.

CI in https://github.com/rapidsai/docker has been failing because of errors in the cuspatial_api_examples.ipynb notebook. Specifically, that notebook refers unconditionally to local files that are not guaranteed to exist.

To fix this, that proposes the following:

• in the notebook, conditionally recreate those files if they don't yet exist
• always test the notebooks in docs/ in CI here

CI here is failing because the nyc_taxi_years_correlation notebook recently started taking a prohibitively long time to run. This proposes:

• skipping nyc_taxi_years_correlation notebook in CI

And some other small things noticed while doing all this:

• fixing typo (rint -> ring) and argument ordering in GeoSeries.from_polygons_xy() docs

Checklist

• I am familiar with the Contributing Guidelines.
• New or existing tests cover these changes.
• The documentation is up to date with these changes.

Notes for Reviewers

I ran these notebooks on a system with CUDA 12.2, using the latest 24.8.* nightlies of RAPIDS conda packages.

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[jameslamb]

(edited Jul 21): moved these details to a separate issue, #1413

---

I think that the nyc_taxi_years_correlation.ipynb notebook has just become prohibitively expensive to run in CI.

• this PR: 2.5+ hours before I killed it (build link)
• most recent successful run (July 11): 8m38s (build link)

Possible reasons I can think of:

• cuspatial or cudf is now falling back to the CPU for some operation it was previously able to do entirely on the GPU
• the input data hosted at those remote storage locations has grown significantly
• some other performance regression in cudf, cuspatial, or their dependencies
• some change in the CI environment

On a machine with CUDA 12.2, latest nightlies of RAPIDS libraries (conda), and 8 V100s, I reduced that notebook down to this minimal example:

imports and data setup (click me)

import cuspatial
import geopandas as gpd
import cudf
import numpy as np

# (shell commands run in notebook cells)
# !if [ ! -f "tzones_lonlat.json" ]; then curl "https://data.cityofnewyork.us/api/geospatial/d3c5-ddgc?method=export&format=GeoJSON" -o tzones_lonlat.json; else echo "tzones_lonlat.json found"; fi
# !if [ ! -f "taxi2016.csv" ]; then curl https://storage.googleapis.com/anaconda-public-data/nyc-taxi/csv/2016/yellow_tripdata_2016-01.csv -o taxi2016.csv; else echo "taxi2016.csv found"; fi   

taxi2016 = cudf.read_csv("taxi2016.csv")
tzones = gpd.GeoDataFrame.from_file('tzones_lonlat.json')
taxi_zones = cuspatial.from_geopandas(tzones).geometry
taxi_zone_rings = cuspatial.GeoSeries.from_polygons_xy(
    taxi_zones.polygons.xy,
    taxi_zones.polygons.ring_offset,
    taxi_zones.polygons.part_offset,
    cudf.Series(range(len(taxi_zones.polygons.part_offset)))
)

def make_geoseries_from_lonlat(lon, lat):
    lonlat = cudf.DataFrame({"lon": lon, "lat": lat}).interleave_columns()
    return cuspatial.GeoSeries.from_points_xy(lonlat)

pickup2016 = make_geoseries_from_lonlat(taxi2016['pickup_longitude'] , taxi2016['pickup_latitude'])
dropoff2016 = make_geoseries_from_lonlat(taxi2016['dropoff_longitude'] , taxi2016['dropoff_latitude'])

pip_iterations = list(np.arange(0, 263, 31))
pip_iterations.append(263)
print(pip_iterations)

# --- begin point-in-polygon calculation ---#
# %%time
taxi2016['PULocationID'] = 264
taxi2016['DOLocationID'] = 264

start = pip_iterations[0]
end = pip_iterations[1]

zone = taxi_zone_rings[start:end]

# find all pickups in that zone
pickups = cuspatial.point_in_polygon(pickup2016, zone)
print(pickups)
print("---")
dropoffs = cuspatial.point_in_polygon(dropoff2016, zone)
print(dropoffs)

Just that bit of code for one combination of polygons took 21 minutes to run. And in the notebook, we're running that for 10 combinations.

cuspatial/notebooks/nyc_taxi_years_correlation.ipynb

Lines 168 to 169 in c8616c1

	"pip_iterations = list(np.arange(0, 263, 31))\n",
	"pip_iterations.append(263)\n",

[0, 31, 62, 93, 124, 155, 186, 217, 248, 263]

cuspatial/notebooks/nyc_taxi_years_correlation.ipynb

Lines 207 to 209 in c8616c1

	"for i in range(len(pip_iterations)-1):\n",
	" start = pip_iterations[i]\n",
	" end = pip_iterations[i+1]\n",

So conservatively, it might take 3.5 hours for the notebook to finish in my setup. And that's making a LOT of assumptions.

[jameslamb]

This -f test was checking a different filename than is actually downloaded, so it re-downloaded the data every time this cell was run.

[jakirkham]

Thanks all! 🙏