Repository navigation
feat(catalog): refuse a catalog newer than the supported schema version - #70
Merged
Merged
Conversation
Catalog load reads catalog_schema_version from the parquet sddb_release_info metadata and raises a clear error when it exceeds SUPPORTED_CATALOG_SCHEMA=2. A legacy catalog that declares no version loads unchanged (missing == OK).
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #70 +/- ##
==========================================
+ Coverage 94.67% 94.72% +0.05%
==========================================
Files 22 22
Lines 2291 2313 +22
==========================================
+ Hits 2169 2191 +22
Misses 122 122 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Per review: only a missing sddb_release_info or a missing catalog_schema_version key is legacy. Provenance that is present must be sound — unparseable sddb_release_info, or a non-integer catalog_schema_version, now raises ValueError (a release shipping broken provenance is a bug), as does an int above SUPPORTED_CATALOG_SCHEMA.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The catalog loader now refuses a catalog cut under a schema newer than this client understands, instead of mis-parsing it.
SUPPORTED_CATALOG_SCHEMA = 2in_catalog_schema.py— the newest schema this client reads.Catalog(...)readscatalog_schema_versionfrom the parquetsddb_release_infometadata (a JSON copy ofrelease_info.jsonthe producer embeds, so it travels with the file through both remote download and the local-copy cache, exactly likesddb_generated_at).> 2raises a clearValueErrornaming the version and pointing atpip install -U spatialdata-db-client.<= 2) catalog and loads unchanged.Why
The engine bumps
catalog_schema_versionto3when the catalog emits vocab technology ids + a vendor column. An old client must fail loud on such a catalog rather than silently drop or misread the new columns. The website does the same guard on its side.Scope note
Only the parquet metadata is read. A separate sibling
release_info.jsonwas considered but rejected:fetch_catalogcopies even local catalogs into a single-file cache, so a sidecar would not survive the fetch; the embedded parquet metadata is the only reliable transport.Tests
tests/test_catalog_metadata.py: supported version loads, absent version loads (legacy), newer version raises. Full catalog suite green (102 passed), ruff clean.