Skip to content

metric sinks: SHOW, catalog visibility, and RBAC (SQL-572) - #38149

Open
mtabebe wants to merge 1 commit into
MaterializeInc:mainfrom
mtabebe:ma/prom-metrics/sql-572-show-rbac
Open

metric sinks: SHOW, catalog visibility, and RBAC (SQL-572)#38149
mtabebe wants to merge 1 commit into
MaterializeInc:mainfrom
mtabebe:ma/prom-metrics/sql-572-show-rbac

Conversation

@mtabebe

@mtabebe mtabebe commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Problem:

A metric sink was invisible, mz_objects did not carry it, and there
was no way to read back the SQL that created it.

Solution:

SHOW METRIC SINKS [FROM <schema>] [IN CLUSTER <c>] [LIKE ...] lists
name, from, and cluster. SHOW [REDACTED] CREATE METRIC SINK replays
create_sql. Both require enable_metric_sink.

mz_internal.mz_metric_sinks exposes what the catalog knows about a
sink, shaped like mz_catalog.mz_sinks, and joins into mz_objects as
type metric-sink. It is a materialized view derived from
mz_catalog_raw. It carries owner_id, so a sink's owner is now
visible in the catalog and its ownership checks are testable.

The relation carries only the columns something reads. create_sql and
redacted_create_sql have no reader, and SHOW CREATE is how you get
the SQL back.

Testing:

  • metric_sink.slt covers discovery and access control: the
    mz_metric_sinks row resolving to its FROM relation, cluster, schema,
    and owner, mz_objects and mz_show_all_objects membership, the three
    SHOW METRIC SINKS filters, the SHOW CREATE round-trip, the audit
    rows, and privileges.

  • The restart platform check reads SHOW METRIC SINKS instead of
    probing with a CREATE expected to fail.

@mtabebe
mtabebe force-pushed the ma/prom-metrics/sql-572-show-rbac branch 2 times, most recently from 2599599 to 5eeeece Compare August 11, 2026 14:35
@mtabebe
mtabebe force-pushed the ma/prom-metrics/sql-572-show-rbac branch 5 times, most recently from f9116fd to 63996e5 Compare August 21, 2026 00:30
Problem:

A metric sink was invisible, `mz_objects` did not carry it, and there
was no way to read back the SQL that created it.

Solution:

`SHOW METRIC SINKS [FROM <schema>] [IN CLUSTER <c>] [LIKE ...]` lists
name, `from`, and cluster. `SHOW [REDACTED] CREATE METRIC SINK` replays
`create_sql`. Both require `enable_metric_sink`.

`mz_internal.mz_metric_sinks` exposes what the catalog knows about a
sink, shaped like `mz_catalog.mz_sinks`, and joins into `mz_objects` as
type `metric-sink`. It is a materialized view derived from
`mz_catalog_raw`. It carries `owner_id`, so a sink's owner is now
visible in the catalog and its ownership checks are testable. Like
`mz_sinks`, it is indexed on `id` so the `mz_objects` union reuses the
arrangement.

The relation carries only the columns something reads. `create_sql` and
`redacted_create_sql` have no reader, and `SHOW CREATE` is how you get
the SQL back.

`enable_metric_sink` gates the DDL and the SHOW verbs, not the catalog
relation. `mz_metric_sinks` is always present and public, so with the
flag off it is simply empty.

Testing:

- `metric_sink.slt` covers discovery and access control: the
`mz_metric_sinks` row resolving to its `FROM` relation, cluster, schema,
and owner, `mz_objects` and `mz_show_all_objects` membership, the three
`SHOW METRIC SINKS` filters, the `SHOW CREATE` round-trip (plain and
redacted), the audit rows, and privileges.

- The restart platform check reads `SHOW METRIC SINKS` instead of
probing with a `CREATE` expected to fail.

Co-authored-by: Moritz Hoffmann <antiguru@gmail.com>
@mtabebe
mtabebe force-pushed the ma/prom-metrics/sql-572-show-rbac branch from 051c708 to f34efdc Compare August 21, 2026 15:06
@mtabebe
mtabebe requested review from SangJunBak and antiguru August 21, 2026 17:11
@mtabebe
mtabebe marked this pull request as ready for review August 21, 2026 17:11
@mtabebe
mtabebe requested review from a team as code owners August 21, 2026 17:11
@def-

def- commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

QA LLM Review

1. HIGH -- new builtin index changes mz_indexes's fingerprint but reuses a migration step from the previous release cycle

src/adapter/src/catalog/open/builtin_schema_migration.rs:398

Adding mz_metric_sinks_ind changes mz_catalog.mz_indexes's SQL fingerprint, but the only mz_indexes replacement step is pinned at 26.39.0-dev.0, which now sorts below the release this change no longer ships in. Upgrading a v26.39.x environment to v26.40.x therefore skips the step, and environmentd aborts at catalog open with fingerprint mismatch for builtin ... mz_indexes.

Details

This branch was written against a tree whose workspace version was 26.39.0-dev.0, so reusing the existing step was correct then. main has since bumped to 26.40.0-dev.0, and v26.39.0-rc.1 is already tagged carrying mz_object_graph_edges_ind but not mz_metric_sinks_ind. After a rebase the step is one release cycle stale, exactly the hazard the NOTE a few steps above (builtin_schema_migration.rs:271) describes.

Mechanism, all in builtin_schema_migration.rs:

  • plan_migration keeps only steps with version > source_version (line 778). semver orders 26.39.0-dev.0 < 26.39.0-rc.1 < 26.39.0, so any v26.39.x source drops it.
  • The force_migration escape hatch that papers over dev-to-dev upgrades only triggers when the source version's pre-release starts with dev (line 671), so a release or rc source takes the version-filtered path.
  • update_fingerprints then sees mz_indexes with a changed fingerprint and no migration, and panics (line 1209). Builtin::fingerprint() for a materialized view is its create_sql, and make_mz_indexes inlines the entire builtin-index set as VALUES precisely so that adding an index is a hard failure rather than silent drift (src/catalog/src/builtin.rs:1502).

Nothing on this PR will go red: fresh bootstraps never compare fingerprints, and the upgrade suites currently start from v26.38.1, from which the 26.39.0-dev.0 step still applies. The failure appears only once v26.39.0 becomes an upgrade source.

Fix: leave the existing step alone (it is historical, and covers only mz_object_graph_edges_ind) and add one at the workspace's current dev version:

        // Required because we added the `mz_metric_sinks_ind` builtin index.
        // make_mz_indexes inlines the builtin-index set as VALUES, so any add
        // or remove changes its SQL fingerprint and requires an explicit
        // replacement. See the NOTE above: this version must stay at the
        // workspace's current dev version until the change ships.
        MigrationStep::replacement(
            "26.40.0-dev.0",
            CatalogItemType::MaterializedView,
            MZ_CATALOG_SCHEMA,
            "mz_indexes",
        ),

The new mz_internal.mz_metric_sinks materialized view itself correctly needs no step: it has no durable mapping in an older catalog, and validate_migration_steps (line 763) would panic on a step naming it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants