Spatial data validations

[tdikland]

Changes

Add specialised data validations for geospatial data.

The following checks are implemented:

• is_longitude
• is_latitude
• is_geometry
• is_geography
• is_point
• is_linestring
• is_polygon
• is_multipoint
• is_multilinestring
• is_multipolygon
• is_geometrycollection
• is_ogc_valid
• is_non_empty_geometry
• has_x_coordinate_between
• has_y_coordinate_between

Linked issues

Resolves #453

Tests

• manually tested
• added unit tests
• added integration tests
• added end-to-end tests
• added performance tests

[github-actions]

All commits in PR should be signed ('git commit -S ...'). See https://docs.github.com/en/authentication/managing-commit-signature-verification/signing-commits

[mwojtyczka]

Please sign all commits

[Copilot]

Pull Request Overview

This PR introduces specialized data validation functions for geospatial data types, implementing validation checks for geometry and geography columns.

• Adds is_valid_geometry and is_valid_geography functions for spatial data validation
• Implements row-level validation using Databricks-specific SQL functions
• Provides comprehensive test coverage for both validation functions

Reviewed Changes

Copilot reviewed 2 out of 3 changed files in this pull request and generated 1 comment.

File	Description
src/databricks/labs/dqx/geo/check_funcs.py	Implements geospatial validation functions using try_to_geometry and try_to_geography
tests/integration/test_row_checks_geo.py	Integration tests validating the behavior of geometry and geography check functions

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[tdikland]

@mwojtyczka Can we do another review? I think the basics are there.

[mwojtyczka]

Please add documentation here: https://github.com/tdikland/dqx/blob/feat/geo/docs/dqx/docs/reference/quality_checks.mdx#row-level-checks-reference

Description of checks functions and examples that use classes and yaml.

Please add integration tests similar to test_apply_checks_all_checks_using_classes and test_apply_checks_all_row_checks_as_yaml_with_streaming.

Please add perf tests: https://github.com/tdikland/dqx/blob/feat/geo/tests/perf/test_apply_checks.py

[Copilot]

Pull Request Overview

Copilot reviewed 2 out of 3 changed files in this pull request and generated 7 comments.

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[mwojtyczka]

LGTM

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.