Conversation
Merges converter output into existing YAML data files by relation name, preserving hand-authored entries (e.g. mz_active_peeks) that are skipped by the converter itself.
…erter Resolving a bare reference-style shortcut (e.g. `[dataflow]`) in a column's `meaning` text turns it into a fully-formed inline link, which lint-docs-catalog.py's comment-text stripper then strips down to bare text. An unresolved shortcut is left bracketed instead, and that bracketed form is what the live SQL comment literally contains, so resolving it broke the pure-move invariant for mz_introspection's migration. Relation-level descriptions aren't checked against SQL comments, so they keep resolving shortcuts as before.
Runs the catalog-docs converter over mz_introspection.md, moving all 25 remaining documented relations into doc/user/data/mz_introspection.yml and doc/user/data/catalog_types.yml, and rewriting their inline tables into catalog-relation shortcode calls. mz_active_peeks (already a data entry) and mz_cluster_prometheus_metrics (NO_COMMENTS) are untouched. Verified as a pure move: ci/test/lint-docs-catalog.sh produces zero diff against the checked-in .slt files, so the generated SQL-comment checks are byte-identical to before the migration.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Phase 2 of the system-catalog docs data migration.
Stacked on phase 1 (base branch
catalog-docs-data-phase1); retarget tomainonce phase 1 lands.Description
Moves the 25 remaining comment-checked
mz_introspectionrelations from inline markdown tables intodoc/user/data/mz_introspection.yml, driven by a reusable converter (ci/test/migrate-docs-catalog.py).This is a pure relocation: the generated
test/sqllogictest/autogenerated/mz_introspection.sltis unchanged, which proves every base-table catalog check stays byte-identical.Column meanings are kept verbatim to match the catalog comments (reference-style shortcut links stay literal in meanings, resolved only in descriptions).
mz_cluster_prometheus_metrics(aNO_COMMENTSsource) stays inline pending a no-comments path;_per_worker/_rawsiblings stay asRELATION_SPEC_UNDOCUMENTEDmarkers pending phases 3 and 4.Verification
ci/test/lint-docs-catalog.shreports zero diff (the pure-move proof),bin/sqllogictestpasses on the regenerated file, andhugo+ link checks are clean. The converter has unit tests (ci/test/test_migrate_docs_catalog.py).