diff --git a/docs/design/curated-dashboard-packs-plan.md b/docs/design/curated-dashboard-packs-plan.md
index 8b6333a8..32f9f863 100644
--- a/docs/design/curated-dashboard-packs-plan.md
+++ b/docs/design/curated-dashboard-packs-plan.md
@@ -79,6 +79,23 @@ tests/
## Key Discoveries (update as you go)
+**PostgreSQL Exporter 12485 (2026-08-31) — see `curated-pack-12485-postgresql-exporter.md`:**
+- The shared curated rig (`parity-rig/curated/grafana_763_redis_exporter/`) already
+ runs a real `prometheuscommunity/postgres-exporter:v0.15.0` + load generator into
+ `metrics-postgres.prometheus-default`. New postgres packs validate here — no
+ throwaway rig. Enable `--collector.stat_statements` + `--collector.postmaster`
+ (+ the `pg_stat_statements` extension) on the rig's postgres services when a pack
+ needs those series; redis/other exporters are untouched.
+- Read the **real** exporter (`curl :9187/metrics`) before writing `metric_map`:
+ 12485 was authored against an older lineage, so `pg_database_size` →
+ `pg_database_size_bytes`, `pg_replication_lag` → `pg_replication_lag_seconds`,
+ `pg_stat_statements_calls` → `_calls_total`, `pg_stat_statements_total_time_seconds`
+ → `_seconds_total`. `pg_stat_activity_count` / `pg_locks_count` are **gauges**
+ despite the `_count` suffix — force them in `metric_kinds`.
+- Live result: 35 panels, 0 Red / 0 not-feasible, uploaded; render audit 0
+ render_error (32/32 rendered); both controls populate + bind (`?Instance` ×57,
+ `?Database` ×23). 14114 re-validated on the same rig: 6/6, render PASS, no regression.
+
**Dashboard investigation (2026-07-29):**
- Dashboard **12776** ("Redis" by the Redis org): uses `redis-datasource` plugin (NOT Prometheus). Queries are Redis commands (`INFO`, `CLIENT LIST`, `SLOWLOG GET`) with empty `expr` fields. `infer_query_language()` returns `"unknown"` for empty queries → panels land as not_feasible. **Not the right target for a PromQL curated pack.** The 12776 readme on grafana.com links to two Prometheus-based alternatives (see below). Future work: a separate ES|QL-injection curated pack variant for `redis-datasource` type dashboards mapped to Elastic Redis integration fields.
- Dashboard **763** ("Redis Dashboard for Prometheus Redis Exporter 1.x" by oliver006): uses `prometheus` datasource. 13 panels, all PromQL using `redis_exporter` metrics. gnetId=763, revision=6 (latest, 2024-02-17, 181,727 downloads). **This is our first target.**
diff --git a/docs/design/curated-dashboard-packs.md b/docs/design/curated-dashboard-packs.md
index 7acf3d6c..52760552 100644
--- a/docs/design/curated-dashboard-packs.md
+++ b/docs/design/curated-dashboard-packs.md
@@ -165,7 +165,15 @@ panel:
status_override: migrated # migrated | migrated_with_warnings (default: migrated)
```
-**Merge semantics:** User pack overrides win by `title_match`. If both the curated pack and the user `--rules-file` declare an override for the same panel title, the user's query wins. Overrides with different titles are merged (both apply).
+Optional `section_match` scopes the override to a Grafana row whose title
+casefolds to (or starts with) that string, the same way layout overrides
+distinguish Global vs Database duplicate titles. Layout matching uses that
+source title even when a layout override later renames the section.
+
+**Merge semantics:** User pack overrides win by `(title_match, section_match)`.
+If both the curated pack and the user `--rules-file` declare an override for
+the same panel title and section, the user's query wins. Overrides with
+different titles or sections are merged (both apply).
**ES|QL shape constraints:** The query must produce a shape that `_native_esql_panel_spec` can parse for the target Kibana panel type:
- `metric` / `gauge` panels: a `STATS` query with exactly one metric column and no `BY` clause. The simplest form is an inline division: `STATS value = MAX(...) / MAX(...)`.
diff --git a/docs/design/curated-pack-12485-postgresql-exporter.md b/docs/design/curated-pack-12485-postgresql-exporter.md
new file mode 100644
index 00000000..2d34e702
--- /dev/null
+++ b/docs/design/curated-pack-12485-postgresql-exporter.md
@@ -0,0 +1,168 @@
+# Curated Pack — Grafana 12485 "PostgreSQL Exporter"
+
+> Design + living discoveries for the 12485 curated pack. Follows the general
+> Curation Playbook in `curated-dashboard-packs-plan.md`. 14114 (PostgreSQL
+> Exporter Quickstart) already ships a validated pack; this doc is the net-new
+> 12485 work plus a live re-validation pass of 14114.
+
+- Source: Grafana Labs / community **"PostgreSQL Exporter"**,
+
+- gnetId **12485**, only revision is **1** (2020-06-17).
+- canonical sha256 (rev 1) = `e14a35eac532db4f79837a293411edb20d04164ca59de17d73557d08637a4700`
+ (matches `parity-rig/benchmark/community_corpus.json`).
+- Datasource: Prometheus (`__inputs[0].pluginId == prometheus`) — curated-pack eligible.
+- Panels: ~37 across two sections — **Global Statistics** and **Database: $Database**
+ (the DB section repeats the same metric families scoped by `datname`).
+- Controls: `Instance`, `Database`, `Interval` (Grafana interval var).
+
+## Goal
+
+Ship a curated pack so 12485 renders in Kibana with the same information as
+Grafana (or better) against a real `postgres_exporter` scrape ingested in the
+Elastic `prometheus_native` layout (`metrics.* + labels.*`), then prove it with
+render + interaction audits and a side-by-side.
+
+## Engine vs pack split
+
+The general pipeline already handles rate/gauge translation, five-target
+fusion, and control synthesis. The pack only carries what 12485 needs beyond
+that, verified against the **real** exporter (not guessed):
+
+### Empirical exporter findings (prometheuscommunity/postgres-exporter v0.15.0)
+
+Read directly from the live rig exporter (`curl :9187/metrics`). The dashboard
+was authored against an older exporter lineage, so several names differ from
+what v0.15.0 actually emits:
+
+| Dashboard PromQL name | Real exporter field | Kind | Note |
+|---|---|---|---|
+| `pg_database_size` | `pg_database_size_bytes` | gauge | rename |
+| `pg_replication_lag` | `pg_replication_lag_seconds` | gauge | rename; 0 on a standalone primary |
+| `pg_stat_statements_calls` | `pg_stat_statements_calls_total` | counter | needs `--collector.stat_statements` + extension |
+| `pg_stat_statements_total_time_seconds` | `pg_stat_statements_seconds_total` | counter | same |
+| `pg_postmaster_start_time_seconds` | *(same)* | gauge | needs `--collector.postmaster` (off by default) |
+| `pg_stat_activity_count` | *(same)* | **gauge** | `_count` suffix would mislead offline heuristic → force gauge |
+| `pg_locks_count` | *(same)* | **gauge** | same |
+| `pg_settings_shared_buffers_bytes` | *(same)* | gauge | present as-is |
+| `pg_stat_database_*` (xact/tup/blk_time/deadlocks/temp_files/blks_*) | *(same)* | counter | exporter `# TYPE` = counter |
+
+`stat_statements` and `postmaster` collectors are **off by default** in v0.15.0,
+and `pg_stat_statements` requires `shared_preload_libraries` + `CREATE EXTENSION`.
+The rig (`parity-rig/curated/grafana_763_redis_exporter/`) was extended to enable
+all three so Query rate / Average query runtime / Uptime render on real data.
+On a target cluster that does not run these, those panels are an honest
+`field_gap`/`data_gap`, not a translator bug.
+
+### Pack rules
+
+- **`metric_kinds`** — force `pg_stat_activity_count` + `pg_locks_count` +
+ `pg_stat_database_numbackends` to `gauge`; assert counters for the rated
+ `pg_stat_database_*` and mapped `pg_stat_statements_*` targets; gauges for the
+ `pg_settings_*` / size / lag / start-time series.
+- **`metric_map`** — the four renames above (targets emitted verbatim under `metrics.`).
+- **`label_rewrites` / `label_candidates`** — `instance`→`labels.instance`,
+ `datname`/`db`→`labels.datname`, `job`→`labels.job`, `state`→`labels.state`,
+ `mode`→`labels.mode`.
+- **`controls.field_overrides`** — `Instance`/`instance`→`labels.instance`,
+ `Database`/`database`/`datname`→`labels.datname` (both cases, since
+ `resolve_control_field` matches the variable name exactly).
+- **`plugin.py`** — rewrite `Instance` populate
+ `label_values({job="postgres-exporter"}, instance)` → `label_values(pg_up, instance)`
+ (the `postgres-exporter` job filter never matches Elastic labels); rewrite
+ `Database` populate `label_values(datname)` →
+ `label_values(pg_stat_database_numbackends, datname)` (needs a per-db metric
+ anchor); drop the `Interval` control if it lands as an inert control.
+
+### Fidelity
+
+- **PERFECT**: rate panels (Transactions, Tuples, Deadlocks, Temp files, I/O
+ time, Transaction rate, Query rate), gauge stats (Version, Max/Shared
+ buffers, Active clients, Connections by state/db, Locks by state, DB size,
+ Replication lag, Numbackends).
+- **APPROXIMATE** (PERFECT under native PROMQL, documented delta in ES|QL):
+ Shared Buffer Hits, Commit Ratio, Connections used, PostgreSQL Uptime
+ (`time() - start_time`). Average query runtime is a curated last-non-null
+ `rate/rate` override so the KPI does not render N/A on the incomplete
+ window-edge `delta/delta` bucket. Add per-panel ES|QL `query_overrides`
+ only where a panel would otherwise `render_error` or empty-state.
+
+## Validation gates (UI testing)
+
+1. Migrate + upload to Kibana (`prometheus_native`, `--esql-index` = data view).
+2. Render audit — 0 `render_error`; any `field_gap`/`data_gap` documented in
+ `fidelity_manifest.yaml`.
+3. Interaction audit — `Instance` + `Database` controls rewrite panel queries.
+4. Side-by-side vs the provisioned Grafana 12485 in a clean Kibana view session.
+
+## Task checklist
+
+- [x] registry.yaml entry (12485, rev 1, sha above)
+- [x] pack.yaml + plugin.py + fidelity_manifest.yaml
+- [x] offline fixture tests; `typecheck` green (own files `ruff`-clean)
+- [x] rig: enable stat_statements + postmaster + extension
+- [x] live: migrate + upload + render/interaction audit
+- [x] re-validate 14114 on the same rig
+- [x] docs: discoveries here + `docs/sources/grafana.md`
+
+## Live validation results (2026-08-31, rig ES 9.5 + Kibana, real postgres_exporter)
+
+**Migration** (`obs-migrate migrate --field-profile prometheus_native --es-url … --upload`):
+35 panels — 18 migrated, 17 migrated_with_warnings, **0 requires-manual, 0
+not-feasible; verification gate 18 Green / 17 Yellow / 0 Red; 35/35 ES|QL
+queries validated; uploaded**. 23 panels native-PROMQL (oracle-verifiable), 12
+ES|QL. Curated pack auto-fired on gnetId 12485.
+
+**Render audit** (`render_audit_driver --elements`, headless Chrome vs the live
+upload): **35 panels in the uploaded dashboard; 0 render_error, 0 error markers,
+0 console/server errors on the expanded tree.** Emitted viz types are correct
+(graphs → `xy`, singlestats → `metric`, ratios → `gauge`). The pack forces the
+`Database:` section open (`collapsed: false`) so Database-scoped panels are
+part of the default view, not hidden behind a collapsed row. Remaining `warn`
+status is the element audit's per-element chart-kind heuristic (XY panels
+expose a secondary metric element), not missing panels. `grafana-validate-uploaded`
+is **35/35** with 0 empty / 0 overlaps / 0 runtime errors.
+
+**Interaction**: both controls populate from live ES —
+`Instance` → `['.*', 'postgres:5432']`, `Database` → `['.*', 'postgres',
+'rigdb', 'template0', 'template1']` — and panels bind the params (`?Instance`
+×57, `?Database` ×23), e.g. `max(metrics.pg_replication_lag_seconds{instance=~?Instance})`.
+
+**Visual**: every panel renders real data (Uptime 16.57 min via the postmaster
+collector; Query rate 25.5 & Avg runtime 2.45 ms via pg_stat_statements; Total
+DB size 62.80 MB via the `_bytes` rename; gauges + xy time-series all correct).
+UI polish (2026-09-01): I/O legends are Read/Write; ratio gauges keep chrome
+titles; Global KPI strip fills 48 cols; Database section is a hole-free 3+2
+KPI grid plus 24+24 graph pairs; Locks by state is a stacked bar with the
+legend on the right; Deadlocks / temp files legend by database name instead of
+a leftover `deadlocks`/`temp_files` series; Replication
+lag spans the full row. Average query runtime skips the incomplete last
+`delta/delta` bucket (native PROMQL `LAST` → Kibana duration N/A) and shows
+last-non-null seconds-per-call (~2.4 ms on the rig).
+
+**14114 re-validation** (same rig, single input): curated pack fired, **6/6
+migrated, 6 Green / 0 Red, render audit PASS (6/6 rendered, 0 errors)**. This
+PR also updates the 14114 pack: Number of active connections now groups by
+`instance` and `datname` (Lens `series_group` composite so two exporters that
+share a database name stay distinct series), plus layout overrides so the
+Quickstart dashboard matches Kibana chart conventions.
+
+## Discoveries
+
+- The curated Redis rig (`parity-rig/curated/grafana_763_redis_exporter/`) is a
+ **shared multi-exporter rig** that already runs a real
+ `prometheuscommunity/postgres-exporter:v0.15.0` + a load generator, ingested
+ via `redis_scraper.py` into `metrics-postgres.prometheus-default`
+ (`prometheus_native`). New postgres packs validate here rather than in a
+ throwaway rig; the scraper honours each exporter's `# TYPE`, so counters land
+ counter-typed in ES.
+- The rig exporter's default collectors do NOT emit
+ `pg_postmaster_start_time_seconds` or any `pg_stat_statements_*`. Enabling
+ `--collector.postmaster` + `--collector.stat_statements` (with
+ `shared_preload_libraries=pg_stat_statements` + `CREATE EXTENSION`) surfaced
+ `pg_postmaster_start_time_seconds` (gauge), `pg_stat_statements_calls_total`
+ and `pg_stat_statements_seconds_total` (counters, labelled by `datname`,
+ `queryid`, `user`).
+- `metric_map` targets are emitted **verbatim** (the field-profile prefix is
+ not prepended), so map targets must include the `metrics.` prefix; non-mapped
+ gauges are emitted bare offline (`pg_stat_activity_count`, not
+ `metrics.pg_stat_activity_count`).
diff --git a/docs/sources/grafana.md b/docs/sources/grafana.md
index 7a3d3d3c..9db0abc0 100644
--- a/docs/sources/grafana.md
+++ b/docs/sources/grafana.md
@@ -133,7 +133,28 @@ picker; use it when a pinned window (commonly 24h hourly bars) renders empty
in Lens on mixed `metrics-*` even though `_query` returns rows.
`panel.layout_overrides` can also set `title` to rename a section or leaf
panel after translation (Grafana's empty first row becomes Kibana
-"Section 1"; a pack can rename it to "Overview"). Grafana 5 singlestat
+"Section 1"; a pack can rename it to "Overview"), `section_match` so a
+duplicate title in Global vs Database gets independent geometry
+(`section_match` is the Grafana row title, the same string `query_overrides`
+use, captured before any section rename), `hide_title`
+to keep metric/gauge chrome titles visible, `kibana_type_override` /
+`xy_mode` to pick the Lens chart (stacked bar for composition-over-time,
+line for rates) without replacing the query, and `legend_position` to move an
+XY legend (`right` for a long categorical breakdown that does not fit under
+the plot). Those last three are **presentation-only** and stay inside the XY
+family: `layout_overrides.kibana_type_override` accepts `line` / `bar` / `area`
+only (a rule pack asking for `metric`, `gauge`, `datatable`, … is rejected at
+load time), because this late pass rewrites `esql.type` / `mode` / `legend`
+without rebuilding the query, while those shapes require different keys
+(`primary`, `metric`, `breakdowns`) — use `query_overrides` (`esql_query` plus
+its own `kibana_type_override`) for an output-shape change. `xy_mode` needs a
+stackable effective type (`bar` / `area`; a Kibana line chart has no stacking
+mode). When a matched panel translated to a non-XY chart, the presentation
+request is skipped and reported as a panel warning (capped at
+`migrated_with_warnings`) rather than emitted as dashboard JSON that
+`docs/dashboards/schema.json` rejects. `query_overrides` accept the same `section_match` so a duplicated
+Global vs Database title can get different ES|QL (for example Global
+deadlocks must not take `?Database`). Grafana 5 singlestat
panels store units on the panel root (`format: bytes` / `s` / `percent`);
those map to Lens bytes, duration, and `%` formats. Helm-flavored community
dashboards (PostgreSQL Database 9628) may also ship a pack `plugin.py` that
@@ -146,6 +167,45 @@ rewrites Instance from Prometheus `up{job=~"postgres.*"}` to
`label_values(pg_up, instance)` — Elastic prometheus_native scrapes store
exporter health as `pg_up`, not scrape `up` — and drops the unused `$job`
control so native PROMQL panels are not left with an empty Instance param.
+The pack also restretches QPS to the Rows height and lays the four remaining
+graphs as a 24+24 grid so the short Grafana singlestat does not leave a hole,
+and replaces the mixin's `{{__name__}}` connections legend (which GROKs to
+`(null)` under native PROMQL) with a per-`(instance, datname)` ES|QL series
+composited into one Lens XY breakdown.
+The PostgreSQL Exporter (12485) pack targets the same exporter family but was
+authored against an older `postgres_exporter` lineage, so its `metric_map`
+bridges four names that changed in `prometheuscommunity/postgres-exporter`
+v0.15 (`pg_database_size` → `pg_database_size_bytes`, `pg_replication_lag` →
+`pg_replication_lag_seconds`, `pg_stat_statements_calls` →
+`pg_stat_statements_calls_total`, `pg_stat_statements_total_time_seconds` →
+`pg_stat_statements_seconds_total`), and its `metric_kinds` force
+`pg_stat_activity_count` / `pg_locks_count` / `pg_stat_database_numbackends` to
+`gauge` (the `_count` suffix would otherwise make the offline heuristic
+`rate()` a gauge). Its plugin repopulates the `Instance` control from
+`label_values(pg_up, instance)` (the source `up{job="postgres-exporter"}` job
+filter never matches an Elastic scrape) and anchors the bare
+`label_values(datname)` `Database` control on `pg_stat_database_numbackends`,
+and the `Interval` Grafana interval variable is dropped rather than emitted as
+an inert control. The source `Database: $Database` row is a Grafana *repeated*
+row driven by a multi-select variable; Kibana cannot repeat panels, so the
+migration emits one expanded Database section with a **single-select** Database
+control (an explicit control warning says so). Selecting a database scopes those
+panels, but several databases rendered side by side is not reproduced — the
+per-database panel fidelity labels describe the selected database, not the
+repetition. Duplicate Global/Database panel titles are laid out with
+`section_match` so the Database header is a hole-free 3+2 KPI grid and the
+composition panels (connections by state, locks by mode) render as stacked
+bars with the lock-mode legend on the right. Grafana's duplicated `blk_read_time` legend on I/O Read/Write time is
+replaced with explicit Read/Write series. Deadlocks and temporary files legend
+by `datname` instead of a leftover metric name. Average query runtime uses last-non-null
+`rate(seconds_total)/rate(calls_total)` instead of native PROMQL
+`LAST(delta/delta)` — the incomplete window-edge bucket is often 0/0, which
+Kibana's duration formatter renders as N/A even while Query rate is populated
+(Grafana's own singlestat also maps a null current value to "N/A"). `pg_stat_statements` / `pg_postmaster_start_time_seconds`
+panels (Query rate, Average query runtime, Uptime) only show data when the
+target exporter runs the `stat_statements` + `postmaster` collectors and the
+`pg_stat_statements` extension is installed; otherwise they degrade to an
+honest field/data gap.
Each pack is registered in `curated_packs/registry.yaml` with a
`gnet_revision` and `dashboard_sha256` — maintainer-verified provenance pins
diff --git a/observability_migration/adapters/source/grafana/curated_packs/grafana_12485_postgresql_exporter/fidelity_manifest.yaml b/observability_migration/adapters/source/grafana/curated_packs/grafana_12485_postgresql_exporter/fidelity_manifest.yaml
new file mode 100644
index 00000000..28f31030
--- /dev/null
+++ b/observability_migration/adapters/source/grafana/curated_packs/grafana_12485_postgresql_exporter/fidelity_manifest.yaml
@@ -0,0 +1,146 @@
+# Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one or more contributor license agreements.
+# SPDX-License-Identifier: Elastic-2.0
+#
+# Fidelity manifest — Grafana dashboard 12485 (PostgreSQL Exporter)
+# https://grafana.com/grafana/dashboards/12485-postgresql-exporter/
+#
+# PERFECT = same information as Grafana on postgres_exporter.
+# APPROXIMATE = documented delta (lifetime ratio, time()-start, ES|QL ratio).
+# Panels marked "requires: pg_stat_statements" / "requires: replica" render on
+# real data only when that target telemetry exists; otherwise they are an
+# honest field/data gap, not a translator bug.
+
+schema_version: 1
+gnet_id: 12485
+gnet_revision: 1
+dashboard_title: "PostgreSQL Exporter"
+maintainer: "community"
+
+panels:
+ # --- Global Statistics ---------------------------------------------------
+ - title: "PostgreSQL Version"
+ fidelity: PERFECT
+ notes: "max(pg_settings_server_version_num) gauge."
+ - title: "Max Replication Lag"
+ fidelity: PERFECT
+ notes: "max(pg_replication_lag → pg_replication_lag_seconds). requires: replica (0 on a standalone primary)."
+ - title: "Active clients"
+ fidelity: PERFECT
+ notes: "sum(pg_stat_activity_count{state=active}); _count is a gauge, forced in metric_kinds."
+ - title: "Shared Buffer Hits"
+ fidelity: APPROXIMATE
+ notes: "Lifetime blks_hit/(blks_hit+blks_read)*100. Native PROMQL preserves the ratio; ES|QL is per-document."
+ - title: "Connections used"
+ fidelity: APPROXIMATE
+ notes: "sum(numbackends)/max(max_connections). Cross-metric ratio — PERFECT under native PROMQL."
+ - title: "Commit Ratio"
+ fidelity: APPROXIMATE
+ notes: "xact_commit/(xact_commit+xact_rollback) lifetime ratio."
+ - title: "PostgreSQL Uptime"
+ fidelity: APPROXIMATE
+ notes: "time()-pg_postmaster_start_time_seconds. requires: --collector.postmaster. NOW()-based EVAL in Kibana."
+ - title: "Transaction rate"
+ fidelity: PERFECT
+ notes: "sum(rate(xact_commit))+sum(rate(xact_rollback))."
+ - title: "Query rate"
+ fidelity: PERFECT
+ notes: "sum(rate(pg_stat_statements_calls → _calls_total)). requires: pg_stat_statements."
+ - title: "Total database size"
+ fidelity: PERFECT
+ notes: "sum(pg_database_size → pg_database_size_bytes)."
+ - title: "Average query runtime"
+ fidelity: APPROXIMATE
+ notes: "rate(seconds_total)/rate(calls_total) last-non-null bucket (Grafana is current of delta/delta, which maps null to N/A). requires: pg_stat_statements. Same seconds-per-call; skips the incomplete window-edge 0/0 that Kibana duration rendered as N/A."
+ - title: "Shared Buffers"
+ fidelity: PERFECT
+ notes: "pg_settings_shared_buffers_bytes gauge."
+ - title: "Max Connections"
+ fidelity: PERFECT
+ notes: "pg_settings_max_connections gauge."
+ - title: "Connections by state (stacked)"
+ fidelity: PERFECT
+ notes: "sum by (state) (pg_stat_activity_count)."
+ - title: "Connections by database (stacked)"
+ fidelity: PERFECT
+ notes: "sum by (datname) (pg_stat_activity_count)."
+ - title: "Transactions"
+ fidelity: PERFECT
+ notes: "rate(xact_commit), rate(xact_rollback)."
+ - title: "Tuples inserts/updates/deletes"
+ fidelity: PERFECT
+ notes: "rate(tup_inserted/updated/deleted)."
+ - title: "I/O Read/Write time"
+ fidelity: PERFECT
+ notes: "rate(blk_read_time), rate(blk_write_time) counters. Curated Read/Write series names (Grafana legendFormat duplicates blk_read_time on both targets)."
+ - title: "Tuples fetched/returned"
+ fidelity: PERFECT
+ notes: "rate(tup_fetched/returned)."
+ - title: "Locks by state"
+ fidelity: PERFECT
+ notes: "sum by (mode) (pg_locks_count); _count is a gauge, forced in metric_kinds. Stacked bar with legend on the right (Grafana is unstacked lines + bottom table legend)."
+ - title: "Deadlocks by database"
+ fidelity: PERFECT
+ notes: "sum by (datname) (rate(pg_stat_database_deadlocks)). Curated ES|QL legends by labels.datname (native PROMQL leftover was the metric name)."
+ - title: "Temporary files by database"
+ fidelity: PERFECT
+ notes: "sum by (datname) (rate(pg_stat_database_temp_files)). Same datname legend as Deadlocks."
+ - title: "Replication lag "
+ fidelity: PERFECT
+ notes: "max(pg_replication_lag → _seconds). requires: replica."
+ # --- Database: $Database (same families scoped by datname) ---------------
+ - title: "Active clients (per database)"
+ fidelity: PERFECT
+ notes: "pg_stat_activity_count{state=active,datname=~$Database}."
+ - title: "Database size"
+ fidelity: PERFECT
+ notes: "pg_database_size{datname=$Database} → pg_database_size_bytes."
+ - title: "Shared Buffer Hits (per database)"
+ fidelity: APPROXIMATE
+ notes: "Lifetime blks_hit ratio scoped by datname."
+ - title: "Commit Ratio (per database)"
+ fidelity: APPROXIMATE
+ notes: "xact ratio scoped by datname."
+ - title: "Transaction rate (per database)"
+ fidelity: PERFECT
+ notes: "rate(xact_commit+rollback) scoped by datname."
+ - title: "Connections by state (per database)"
+ fidelity: PERFECT
+ notes: "sum by (state) scoped by datname."
+ - title: "Transactions (per database)"
+ fidelity: PERFECT
+ notes: "rate(xact_commit/rollback) scoped by datname."
+ - title: "Tuples inserts/updates/deletes (per database)"
+ fidelity: PERFECT
+ notes: "rate(tup_*) scoped by datname."
+ - title: "Tuples fetched/returned (per database)"
+ fidelity: PERFECT
+ notes: "rate(tup_fetched/returned) scoped by datname."
+ - title: "Locks by state (per database)"
+ fidelity: PERFECT
+ notes: "sum by (mode) (pg_locks_count) scoped by datname. Same stacked bar + right legend as Global."
+ - title: "Deadlocks by database (per database)"
+ fidelity: PERFECT
+ notes: "rate(deadlocks) scoped by datname. Legend is the database name."
+ - title: "Temporary files by database (per database)"
+ fidelity: PERFECT
+ notes: "rate(temp_files) scoped by datname. Legend is the database name."
+
+summary:
+ total_panels: 35
+ perfect: 28
+ approximate: 7
+ not_feasible: 0
+ overall_fidelity: HIGH
+ known_gaps:
+ - "Grafana $Interval interval variable is dropped (not a Kibana control)."
+ - "Instance populate rewritten from up{job=postgres-exporter} to pg_up; Database anchored on pg_stat_database_numbackends."
+ # The source row `Database: $Database` carries repeat=Database on a
+ # multi-select (multi + includeAll) variable. Kibana has no panel
+ # repetition, so the engine emits ONE expanded section with a
+ # single-select Database control (and says so in a control warning).
+ # The 13 per-database panels below are PERFECT for the selected
+ # database; they are not a reproduction of Grafana's repeated rows.
+ - "Grafana repeats the 'Database: $Database' row per selected database; Kibana cannot repeat panels, so the migration emits one expanded Database section with a single-select Database control. Selecting one database scopes those panels; several databases side by side is not reproduced."
+ - "pg_database_size/pg_replication_lag/pg_stat_statements_* renamed to postgres_exporter v0.15 fields via metric_map."
+ - "pg_stat_activity_count and pg_locks_count are gauges despite the _count suffix."
+ - "Replication lag and pg_stat_statements panels require replica / the extension to show non-empty data."
diff --git a/observability_migration/adapters/source/grafana/curated_packs/grafana_12485_postgresql_exporter/pack.yaml b/observability_migration/adapters/source/grafana/curated_packs/grafana_12485_postgresql_exporter/pack.yaml
new file mode 100644
index 00000000..40dfd759
--- /dev/null
+++ b/observability_migration/adapters/source/grafana/curated_packs/grafana_12485_postgresql_exporter/pack.yaml
@@ -0,0 +1,329 @@
+# Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one or more contributor license agreements.
+# SPDX-License-Identifier: Elastic-2.0
+#
+# Curated pack — Grafana dashboard 12485
+# https://grafana.com/grafana/dashboards/12485-postgresql-exporter/
+#
+# Source: community "PostgreSQL Exporter" (Prometheus / postgres_exporter).
+# Revision 1. ~37 panels across a Global Statistics section and a per-database
+# "Database: $Database" section (same metric families scoped by datname).
+#
+# Engine vs pack split (do not duplicate engine work here):
+# Engine: rate()/irate() counter handling, gauge sum/max, five-target fusion,
+# native PROMQL for the ratio panels, control synthesis.
+# Pack: 12485 was authored against the older exporter lineage, so its PromQL
+# names differ from prometheuscommunity/postgres-exporter v0.15
+# (verified live against the parity rig). metric_map bridges the four
+# renames; metric_kinds forces the two `_count` gauges that the offline
+# suffix heuristic would otherwise mistype as counters; plugin.py
+# repopulates the Instance/Database controls (the source `up{job=...}`
+# and bare `label_values(datname)` do not resolve on Elastic scrapes).
+
+query:
+ metrics_dataset_filter: "prometheus"
+
+ # prometheus_native stores every Prometheus label under labels.. Pin the
+ # source labels so offline migrations resolve without a live field-caps probe.
+ label_rewrites:
+ instance: labels.instance
+ job: labels.job
+ datname: labels.datname
+ db: labels.datname
+ state: labels.state
+ mode: labels.mode
+
+ # --- counter / gauge classification -----------------------------------
+ # postgres_exporter v0.15 already declares these via `# TYPE`, but the offline
+ # translation path (no --es-url) needs the classification too, and the two
+ # `_count` gauges MUST be forced or the suffix heuristic makes them counters
+ # (rate() of a gauge → wrong series and IRATE type errors on Elastic).
+ metric_kinds:
+ # Gauges that look like counters by name — the important correctness fix.
+ pg_stat_activity_count: gauge
+ pg_locks_count: gauge
+ pg_stat_database_numbackends: gauge
+ # Plain gauges.
+ pg_up: gauge
+ pg_database_size: gauge
+ pg_database_size_bytes: gauge
+ pg_replication_lag: gauge
+ pg_replication_lag_seconds: gauge
+ pg_postmaster_start_time_seconds: gauge
+ pg_settings_server_version_num: gauge
+ pg_settings_max_connections: gauge
+ pg_settings_shared_buffers_bytes: gauge
+ # Counters (rate()/delta() applied, or lifetime ratio numerators/denominators).
+ pg_stat_database_xact_commit: counter
+ pg_stat_database_xact_rollback: counter
+ pg_stat_database_tup_fetched: counter
+ pg_stat_database_tup_returned: counter
+ pg_stat_database_tup_inserted: counter
+ pg_stat_database_tup_updated: counter
+ pg_stat_database_tup_deleted: counter
+ pg_stat_database_blks_hit: counter
+ pg_stat_database_blks_read: counter
+ pg_stat_database_blk_read_time: counter
+ pg_stat_database_blk_write_time: counter
+ pg_stat_database_deadlocks: counter
+ pg_stat_database_temp_files: counter
+ pg_stat_statements_calls: counter
+ pg_stat_statements_calls_total: counter
+ pg_stat_statements_total_time_seconds: counter
+ pg_stat_statements_seconds_total: counter
+
+ # metric_map targets are emitted VERBATIM (the field profile prefix is NOT
+ # prepended). See docs/command-contract.md, "metric_map targets are verbatim".
+ # These are the four names that changed between the dashboard's exporter
+ # lineage and prometheuscommunity/postgres-exporter v0.15 (verified live).
+ metric_map:
+ pg_database_size: metrics.pg_database_size_bytes
+ pg_replication_lag: metrics.pg_replication_lag_seconds
+ pg_stat_statements_calls: metrics.pg_stat_statements_calls_total
+ pg_stat_statements_total_time_seconds: metrics.pg_stat_statements_seconds_total
+
+ label_candidates:
+ instance:
+ - labels.instance
+ - service.instance.id
+ - host.name
+ job:
+ - labels.job
+ - service.name
+ datname:
+ - labels.datname
+ - datname
+ db:
+ - labels.datname
+ - datname
+ state:
+ - labels.state
+ - state
+ mode:
+ - labels.mode
+ - mode
+
+controls:
+ field_overrides:
+ # resolve_control_field matches the variable name exactly; 12485 uses
+ # capitalised variable names ($Instance / $Database), so pin both cases.
+ Instance: labels.instance
+ instance: labels.instance
+ Database: labels.datname
+ database: labels.datname
+ datname: labels.datname
+
+panel:
+ query_overrides:
+ # Grafana ships both series as legendFormat ``blk_read_time`` (a source
+ # typo on the write target). Give Kibana distinct Read/Write names.
+ - title_match: "I/O Read/Write time"
+ esql_query: |
+ TS metrics-*
+ | WHERE @timestamp >= ?_tstart AND @timestamp <= ?_tend
+ | WHERE (?Instance == "" OR ({{label:instance}} RLIKE ?Instance OR ({{label:instance}} IS NULL AND "" RLIKE ?Instance)))
+ | WHERE {{metric:pg_stat_database_blk_read_time:counter}} IS NOT NULL OR {{metric:pg_stat_database_blk_write_time:counter}} IS NOT NULL
+ | STATS Read = SUM(RATE({{metric:pg_stat_database_blk_read_time:counter}})), Write = SUM(RATE({{metric:pg_stat_database_blk_write_time:counter}})) BY time_bucket = TBUCKET(20, ?_tstart, ?_tend)
+ | KEEP time_bucket, Read, Write
+ | SORT time_bucket ASC
+ status_override: migrated
+ # Native PROMQL LAST(delta/delta) lands on the incomplete window-edge
+ # bucket, which is often 0/0 → NaN. Kibana's duration formatter then
+ # renders N/A even though earlier buckets are ~2.4ms and Query rate
+ # (rate()) is still populated. Grafana's own singlestat maps a null
+ # current value to "N/A"; skip the null last rate bucket the same way
+ # Node Exporter "CPU Busy" does. rate(time)/rate(calls) is the same
+ # seconds-per-call as delta(time)/delta(calls).
+ - title_match: "Average query runtime"
+ esql_query: |
+ TS metrics-*
+ | WHERE @timestamp >= ?_tstart AND @timestamp <= ?_tend
+ | WHERE (?Instance == "" OR ({{label:instance}} RLIKE ?Instance OR ({{label:instance}} IS NULL AND "" RLIKE ?Instance)))
+ | WHERE {{metric:pg_stat_statements_seconds_total:counter}} IS NOT NULL OR {{metric:pg_stat_statements_calls_total:counter}} IS NOT NULL
+ | STATS seconds = SUM(RATE({{metric:pg_stat_statements_seconds_total:counter}})), calls = SUM(RATE({{metric:pg_stat_statements_calls_total:counter}})) BY time_bucket = TBUCKET(20, ?_tstart, ?_tend)
+ | EVAL computed_value = CASE(calls > 0, seconds / calls, NULL)
+ | WHERE computed_value IS NOT NULL
+ | SORT time_bucket DESC
+ | LIMIT 2
+ | SORT time_bucket ASC
+ | LIMIT 1
+ | KEEP computed_value
+ primary_format: s
+ status_override: migrated
+ # Native PROMQL legends these "should be 0" rates as the metric name
+ # (``deadlocks`` / ``temp_files``) instead of ``{{datname}}``. Plot the
+ # rate by ``labels.datname``. Global must not take ?Database (source
+ # query has no datname matcher); the Database section does.
+ - title_match: "Deadlocks by database"
+ section_match: "Global Statistics"
+ kibana_type_override: line
+ esql_query: |
+ TS metrics-*
+ | WHERE @timestamp >= ?_tstart AND @timestamp <= ?_tend
+ | WHERE (?Instance == "" OR ({{label:instance}} RLIKE ?Instance OR ({{label:instance}} IS NULL AND "" RLIKE ?Instance)))
+ | WHERE {{metric:pg_stat_database_deadlocks:counter}} IS NOT NULL
+ | STATS deadlocks = SUM(RATE({{metric:pg_stat_database_deadlocks:counter}})) BY time_bucket = TBUCKET(20, ?_tstart, ?_tend), {{label:datname}}
+ | KEEP time_bucket, `labels.datname`, deadlocks
+ | SORT time_bucket ASC
+ status_override: migrated
+ - title_match: "Deadlocks by database"
+ section_match: "Database"
+ kibana_type_override: line
+ esql_query: |
+ TS metrics-*
+ | WHERE @timestamp >= ?_tstart AND @timestamp <= ?_tend
+ | WHERE (?Instance == "" OR ({{label:instance}} RLIKE ?Instance OR ({{label:instance}} IS NULL AND "" RLIKE ?Instance)))
+ | WHERE (?Database == "" OR ({{label:datname}} RLIKE ?Database OR ({{label:datname}} IS NULL AND "" RLIKE ?Database)))
+ | WHERE {{metric:pg_stat_database_deadlocks:counter}} IS NOT NULL
+ | STATS deadlocks = SUM(RATE({{metric:pg_stat_database_deadlocks:counter}})) BY time_bucket = TBUCKET(20, ?_tstart, ?_tend), {{label:datname}}
+ | KEEP time_bucket, `labels.datname`, deadlocks
+ | SORT time_bucket ASC
+ status_override: migrated
+ - title_match: "Temporary files by database"
+ section_match: "Global Statistics"
+ kibana_type_override: line
+ esql_query: |
+ TS metrics-*
+ | WHERE @timestamp >= ?_tstart AND @timestamp <= ?_tend
+ | WHERE (?Instance == "" OR ({{label:instance}} RLIKE ?Instance OR ({{label:instance}} IS NULL AND "" RLIKE ?Instance)))
+ | WHERE {{metric:pg_stat_database_temp_files:counter}} IS NOT NULL
+ | STATS temp_files = SUM(RATE({{metric:pg_stat_database_temp_files:counter}})) BY time_bucket = TBUCKET(20, ?_tstart, ?_tend), {{label:datname}}
+ | KEEP time_bucket, `labels.datname`, temp_files
+ | SORT time_bucket ASC
+ status_override: migrated
+ - title_match: "Temporary files by database"
+ section_match: "Database"
+ kibana_type_override: line
+ esql_query: |
+ TS metrics-*
+ | WHERE @timestamp >= ?_tstart AND @timestamp <= ?_tend
+ | WHERE (?Instance == "" OR ({{label:instance}} RLIKE ?Instance OR ({{label:instance}} IS NULL AND "" RLIKE ?Instance)))
+ | WHERE (?Database == "" OR ({{label:datname}} RLIKE ?Database OR ({{label:datname}} IS NULL AND "" RLIKE ?Database)))
+ | WHERE {{metric:pg_stat_database_temp_files:counter}} IS NOT NULL
+ | STATS temp_files = SUM(RATE({{metric:pg_stat_database_temp_files:counter}})) BY time_bucket = TBUCKET(20, ?_tstart, ?_tend), {{label:datname}}
+ | KEEP time_bucket, `labels.datname`, temp_files
+ | SORT time_bucket ASC
+ status_override: migrated
+ layout_overrides:
+ # Grafana 24-col leaves holes at x=4 and x=20 on the KPI strip between
+ # Total database size and Average query runtime. Spread the four unique
+ # tiles across 48 cols; keep transformed y/h so we do not open a gap
+ # above the graph rows.
+ - title_match: "Total database size"
+ position: {x: 0}
+ size: {w: 12}
+ - title_match: "Average query runtime"
+ position: {x: 12}
+ size: {w: 12}
+ - title_match: "Shared Buffers"
+ position: {x: 24}
+ size: {w: 12}
+ - title_match: "Max Connections"
+ position: {x: 36}
+ size: {w: 12}
+ # Arc gauges hid the chrome title because the inner label repeats it;
+ # show the chrome title so the three ratio tiles scan like the KPI row.
+ - title_match: "Shared Buffer Hits"
+ hide_title: false
+ - title_match: "Connections used"
+ hide_title: false
+ - title_match: "Commit Ratio"
+ hide_title: false
+ # Per-database panels are the second half of this dashboard; keep them
+ # open so the Database control has something visible to rewrite.
+ - title_match: "Database"
+ collapsed: false
+ # Composition-over-time: stacked bar is the Kibana fit (Grafana already
+ # does this for Connections; Locks was a line of modes).
+ - title_match: "Locks by state"
+ kibana_type_override: bar
+ xy_mode: stacked
+ legend_position: right
+ # Grafana leaves the right half of this row empty.
+ - title_match: "Replication lag"
+ position: {x: 0, y: 84}
+ size: {w: 48, h: 16}
+ # Same 24+16 graph rhythm as the Database section.
+ - title_match: "Connections by state (stacked)"
+ section_match: "Global Statistics"
+ position: {x: 0, y: 20}
+ size: {w: 24, h: 16}
+ - title_match: "Connections by database (stacked)"
+ position: {x: 24, y: 20}
+ size: {w: 24, h: 16}
+ - title_match: "Transactions"
+ section_match: "Global Statistics"
+ position: {x: 0, y: 36}
+ size: {w: 24, h: 16}
+ - title_match: "Tuples inserts/updates/deletes"
+ section_match: "Global Statistics"
+ position: {x: 24, y: 36}
+ size: {w: 24, h: 16}
+ - title_match: "I/O Read/Write time"
+ position: {x: 0, y: 52}
+ size: {w: 24, h: 16}
+ - title_match: "Tuples fetched/returned"
+ section_match: "Global Statistics"
+ position: {x: 24, y: 52}
+ size: {w: 24, h: 16}
+ - title_match: "Locks by state"
+ section_match: "Global Statistics"
+ position: {x: 0, y: 68}
+ size: {w: 24, h: 16}
+ - title_match: "Deadlocks by database"
+ section_match: "Global Statistics"
+ position: {x: 24, y: 68}
+ size: {w: 24, h: 8}
+ - title_match: "Temporary files by database"
+ section_match: "Global Statistics"
+ position: {x: 24, y: 76}
+ size: {w: 24, h: 8}
+ # Database section repeats Global titles; scope these so Global geometry
+ # stays on the faithful transform.
+ - title_match: "Active clients"
+ section_match: "Database"
+ position: {x: 0, y: 0}
+ size: {w: 16, h: 8}
+ - title_match: "Database size"
+ section_match: "Database"
+ position: {x: 16, y: 0}
+ size: {w: 16, h: 8}
+ - title_match: "Transaction rate"
+ section_match: "Database"
+ position: {x: 32, y: 0}
+ size: {w: 16, h: 8}
+ - title_match: "Shared Buffer Hits"
+ section_match: "Database"
+ position: {x: 0, y: 8}
+ size: {w: 24, h: 10}
+ - title_match: "Commit Ratio"
+ section_match: "Database"
+ position: {x: 24, y: 8}
+ size: {w: 24, h: 10}
+ - title_match: "Connections by state (stacked)"
+ section_match: "Database"
+ position: {x: 0, y: 18}
+ size: {w: 24, h: 16}
+ - title_match: "Transactions"
+ section_match: "Database"
+ position: {x: 24, y: 18}
+ size: {w: 24, h: 16}
+ - title_match: "Tuples inserts/updates/deletes"
+ section_match: "Database"
+ position: {x: 0, y: 34}
+ size: {w: 24, h: 16}
+ - title_match: "Tuples fetched/returned"
+ section_match: "Database"
+ position: {x: 24, y: 34}
+ size: {w: 24, h: 16}
+ - title_match: "Locks by state"
+ section_match: "Database"
+ position: {x: 0, y: 50}
+ size: {w: 24, h: 16}
+ - title_match: "Deadlocks by database"
+ section_match: "Database"
+ position: {x: 24, y: 50}
+ size: {w: 24, h: 8}
+ - title_match: "Temporary files by database"
+ section_match: "Database"
+ position: {x: 24, y: 58}
+ size: {w: 24, h: 8}
diff --git a/observability_migration/adapters/source/grafana/curated_packs/grafana_12485_postgresql_exporter/plugin.py b/observability_migration/adapters/source/grafana/curated_packs/grafana_12485_postgresql_exporter/plugin.py
new file mode 100644
index 00000000..3d06f829
--- /dev/null
+++ b/observability_migration/adapters/source/grafana/curated_packs/grafana_12485_postgresql_exporter/plugin.py
@@ -0,0 +1,57 @@
+# Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one or more contributor license agreements.
+# SPDX-License-Identifier: Elastic-2.0
+
+"""Grafana 12485 (PostgreSQL Exporter) curated pack plugin.
+
+Three control-populate rewrites that the general pipeline cannot infer:
+
+* ``Instance`` is ``label_values({job="postgres-exporter"}, instance)``. The
+ ``postgres-exporter`` job filter never matches an Elastic prometheus_native
+ scrape (the job label there is ``postgres_exporter``/whatever the collector
+ sets), so the control would populate empty. Rewrite it to
+ ``label_values(pg_up, instance)`` — the same source-faithful ``instance``
+ label, anchored on a metric that always exists per server.
+* ``Database`` is a bare ``label_values(datname)`` with no metric anchor, so
+ there is nothing for the ES|QL control query to key ``labels.datname`` off.
+ Anchor it on a per-database gauge (``pg_stat_database_numbackends``).
+* ``Interval`` is a Grafana *interval* variable (rate-window helper), not a
+ query variable. It must never become a Kibana control.
+"""
+
+
+_PACK_NAME = "grafana_12485_postgresql_exporter"
+
+
+def register(api):
+ @api["variable_translators"].register("grafana_12485_pg_controls", priority=5)
+ def rewrite_pg_controls(context):
+ pack = getattr(context, "rule_pack", None)
+ if getattr(pack, "_curated_pack_name", "") != _PACK_NAME:
+ return None
+ variable = context.variable or {}
+ name = str(variable.get("name") or "")
+ lname = name.lower()
+ var_type = str(variable.get("type") or "").lower()
+ query_text = context.query_text or str(variable.get("query") or "")
+ compact = query_text.replace(" ", "").lower()
+
+ # Interval helper variable — drop it, no panel binds it as a control.
+ if var_type == "interval" or lname == "interval":
+ context.handled = True
+ return f"skipped Grafana interval variable {name}"
+
+ if lname == "instance" and "label_values(" in compact:
+ rewritten = "label_values(pg_up, instance)"
+ context.query_text = rewritten
+ context.variable = dict(variable)
+ context.variable["query"] = rewritten
+ return None
+
+ if lname in {"database", "datname"} and "label_values(" in compact:
+ rewritten = "label_values(pg_stat_database_numbackends, datname)"
+ context.query_text = rewritten
+ context.variable = dict(variable)
+ context.variable["query"] = rewritten
+ return None
+
+ return None
diff --git a/observability_migration/adapters/source/grafana/curated_packs/grafana_14114_postgres_exporter_quickstart/fidelity_manifest.yaml b/observability_migration/adapters/source/grafana/curated_packs/grafana_14114_postgres_exporter_quickstart/fidelity_manifest.yaml
index 7165424d..d3c722ba 100644
--- a/observability_migration/adapters/source/grafana/curated_packs/grafana_14114_postgres_exporter_quickstart/fidelity_manifest.yaml
+++ b/observability_migration/adapters/source/grafana/curated_packs/grafana_14114_postgres_exporter_quickstart/fidelity_manifest.yaml
@@ -33,17 +33,16 @@ panels:
fidelity: APPROXIMATE
notes: "Grafana divides lifetime blks_hit / (blks_read + blks_hit) without rate(). Native PROMQL preserves that lifetime ratio."
- title: "Number of active connections"
- fidelity: APPROXIMATE
+ fidelity: PERFECT
notes: >
- Native PROMQL of pg_stat_database_numbackends returns the gauge series.
- Grafana legend {{__name__}} GROKs native PROMQL _timeseries, which does
- not carry __name__, so Kibana shows "(null)" in the legend. The line is
- the connection count.
+ Curated ES|QL MAX(LAST_OVER_TIME(numbackends)) BY time_bucket, instance, datname.
+ Grafana {{__name__}} legend is not used; Kibana composites instance / datname
+ into one Lens XY breakdown.
summary:
total_panels: 6
- perfect: 4
- approximate: 2
+ perfect: 5
+ approximate: 1
not_feasible: 0
overall_fidelity: HIGH
known_gaps:
diff --git a/observability_migration/adapters/source/grafana/curated_packs/grafana_14114_postgres_exporter_quickstart/pack.yaml b/observability_migration/adapters/source/grafana/curated_packs/grafana_14114_postgres_exporter_quickstart/pack.yaml
index f80860b6..670f46cf 100644
--- a/observability_migration/adapters/source/grafana/curated_packs/grafana_14114_postgres_exporter_quickstart/pack.yaml
+++ b/observability_migration/adapters/source/grafana/curated_packs/grafana_14114_postgres_exporter_quickstart/pack.yaml
@@ -86,3 +86,46 @@ controls:
db: labels.datname
datname: labels.datname
job: labels.job
+
+panel:
+ query_overrides:
+ # Mixin legendFormat is ``{{__name__}}``. Native PROMQL GROKs that into a
+ # ``breakdown_by: __name__`` column that renders as ``(null)`` in Lens.
+ # Plot numbackends as a named series, split by (instance, datname) so two
+ # exporters that share a database name stay distinct. Lens XY has one
+ # breakdown; the engine composites those labels into ``series_group``.
+ - title_match: "Number of active connections"
+ kibana_type_override: line
+ esql_query: |
+ TS metrics-*
+ | WHERE @timestamp >= ?_tstart AND @timestamp <= ?_tend
+ | WHERE (?instance == "" OR ({{label:instance}} RLIKE ?instance OR ({{label:instance}} IS NULL AND "" RLIKE ?instance)))
+ | WHERE (?db == "" OR ({{label:datname}} RLIKE ?db OR ({{label:datname}} IS NULL AND "" RLIKE ?db)))
+ | WHERE {{metric:pg_stat_database_numbackends:gauge}} IS NOT NULL
+ | STATS connections = MAX(LAST_OVER_TIME({{metric:pg_stat_database_numbackends:gauge}})) BY time_bucket = TBUCKET(20, ?_tstart, ?_tend), {{label:instance}}, {{label:datname}}
+ | KEEP time_bucket, `labels.instance`, `labels.datname`, connections
+ | SORT time_bucket ASC
+ status_override: migrated
+ layout_overrides:
+ # Grafana QPS is a short singlestat beside a tall Rows graph, which leaves
+ # a hole under the tile. Make a consistent 40+8 / 24+24 grid with equal
+ # graph heights so the 48-col canvas has no leftover bands.
+ - title_match: "Rows"
+ position: {x: 0, y: 0}
+ size: {w: 40, h: 14}
+ - title_match: "QPS"
+ position: {x: 40, y: 0}
+ size: {w: 8, h: 14}
+ hide_title: false
+ - title_match: "Buffers"
+ position: {x: 0, y: 14}
+ size: {w: 24, h: 14}
+ - title_match: "Conflicts/Deadlocks"
+ position: {x: 24, y: 14}
+ size: {w: 24, h: 14}
+ - title_match: "Cache hit ratio"
+ position: {x: 0, y: 28}
+ size: {w: 24, h: 14}
+ - title_match: "Number of active connections"
+ position: {x: 24, y: 28}
+ size: {w: 24, h: 14}
diff --git a/observability_migration/adapters/source/grafana/curated_packs/registry.yaml b/observability_migration/adapters/source/grafana/curated_packs/registry.yaml
index ae5ce52b..dee83d4f 100644
--- a/observability_migration/adapters/source/grafana/curated_packs/registry.yaml
+++ b/observability_migration/adapters/source/grafana/curated_packs/registry.yaml
@@ -118,3 +118,12 @@ packs:
gnet_revision: 1
dashboard_sha256: "76b92bbeb9b2d8f3f8abec10b7cb016da87803deca34a99dddab9f668010a53d"
description: "PostgreSQL Exporter mixin — pg_up Instance populate, unused job drop, bgwriter _total map"
+
+ - gnet_id: 12485
+ name: grafana_12485_postgresql_exporter
+ title_hint: "PostgreSQL Exporter"
+ tags_hint: []
+ path: grafana_12485_postgresql_exporter
+ gnet_revision: 1
+ dashboard_sha256: "e14a35eac532db4f79837a293411edb20d04164ca59de17d73557d08637a4700"
+ description: "postgres_exporter overview — v0.15 name remap (size_bytes/lag_seconds/stat_statements), _count gauge fixes, Instance/Database populate rewrite"
diff --git a/observability_migration/adapters/source/grafana/extension_schema.py b/observability_migration/adapters/source/grafana/extension_schema.py
index b8b62a66..703185e0 100644
--- a/observability_migration/adapters/source/grafana/extension_schema.py
+++ b/observability_migration/adapters/source/grafana/extension_schema.py
@@ -8,7 +8,27 @@
import re
from typing import Any
-from pydantic import AliasChoices, BaseModel, ConfigDict, Field, ValidationError, field_validator
+from pydantic import (
+ AliasChoices,
+ BaseModel,
+ ConfigDict,
+ Field,
+ ValidationError,
+ field_validator,
+ model_validator,
+)
+
+# Kibana chart types a *late* ``layout_overrides`` change may pick. The layout
+# pass runs after translation and only rewrites ``esql.type``/``mode``/
+# ``legend``, so it can only move a panel inside the XY family, whose schema
+# definitions (``ESQL{Line,Bar,Area}PanelConfig``) share the same required
+# ``query`` + ``metrics`` shape. Switching to metric/gauge/datatable/pie needs a
+# different set of required keys (``primary``, ``metric``, ``breakdowns``) built
+# from a different query, which is what ``query_overrides`` is for.
+XY_LAYOUT_CHART_TYPES = ("area", "bar", "line")
+# XY types whose schema definition carries a stacking ``mode``. ``line`` does
+# not (``ESQLLinePanelConfig`` declares ``additionalProperties: false``).
+XY_STACKABLE_CHART_TYPES = ("area", "bar")
QUERY_OVERRIDE_FIELDS = {
"default_rate_window",
@@ -65,6 +85,9 @@ class PanelQueryOverrideModel(_StrictModel):
title_match: str
esql_query: str
status_override: str = "migrated"
+ # When set, only apply inside a section whose title casefolds to this
+ # value (or starts with it). Same meaning as layout ``section_match``.
+ section_match: str | None = None
# Optional Lens presentation override when the curated ES|QL shape does
# not match the Grafana panel type (e.g. multi-value bargauge → datatable).
kibana_type_override: str | None = None
@@ -97,9 +120,74 @@ class PanelSizeOverrideModel(_StrictModel):
class PanelLayoutOverrideModel(_StrictModel):
title_match: str
title: str | None = None
+ # When set, only apply inside a section whose title casefolds to this
+ # value (or starts with it). Lets Global vs Database duplicate titles
+ # get independent geometry.
+ section_match: str | None = None
position: PanelPositionOverrideModel = Field(default_factory=PanelPositionOverrideModel)
size: PanelSizeOverrideModel = Field(default_factory=PanelSizeOverrideModel)
collapsed: bool | None = None
+ # When False, keep the Kibana chrome title visible even if the display
+ # mapper hid it (metric/gauge tiles stamp the title onto the inner label).
+ hide_title: bool | None = None
+ # Late *presentation-only* override on the translated ``esql.type``, limited
+ # to the XY family (``line`` / ``bar`` / ``area``) because this pass does not
+ # rebuild the query or the emitted columns. Use
+ # ``query_overrides.kibana_type_override`` to emit a different panel shape.
+ kibana_type_override: str | None = None
+ # Stacking for bar/area (``stacked`` / ``unstacked`` / ``percentage``).
+ # ``line`` has no stacking mode.
+ xy_mode: str | None = None
+ # Late override of the XY legend placement (``bottom`` / ``right`` / …).
+ legend_position: str | None = None
+
+ @field_validator("kibana_type_override")
+ @classmethod
+ def validate_kibana_type_override(cls, value: str | None) -> str | None:
+ if value is None:
+ return value
+ if value not in XY_LAYOUT_CHART_TYPES:
+ raise ValueError(
+ "layout_overrides.kibana_type_override is presentation-only and must "
+ f"be one of {list(XY_LAYOUT_CHART_TYPES)}, got {value!r}; a "
+ "metric/gauge/datatable/pie panel needs different required keys built "
+ "from a different query, so use panel.query_overrides "
+ "(esql_query + kibana_type_override) for that shape change"
+ )
+ return value
+
+ @field_validator("xy_mode")
+ @classmethod
+ def validate_xy_mode(cls, value: str | None) -> str | None:
+ if value is None:
+ return value
+ allowed = {"stacked", "unstacked", "percentage"}
+ if value not in allowed:
+ raise ValueError(f"xy_mode must be one of {sorted(allowed)}, got {value!r}")
+ return value
+
+ @model_validator(mode="after")
+ def validate_xy_mode_target(self) -> PanelLayoutOverrideModel:
+ if self.xy_mode and self.kibana_type_override == "line":
+ raise ValueError(
+ f"xy_mode {self.xy_mode!r} cannot apply to kibana_type_override "
+ "'line': Kibana line charts have no stacking mode. Use "
+ f"kibana_type_override of {list(XY_STACKABLE_CHART_TYPES)}, or drop "
+ "xy_mode"
+ )
+ return self
+
+ @field_validator("legend_position")
+ @classmethod
+ def validate_legend_position(cls, value: str | None) -> str | None:
+ if value is None:
+ return value
+ allowed = {"bottom", "left", "right", "top"}
+ if value not in allowed:
+ raise ValueError(
+ f"legend_position must be one of {sorted(allowed)}, got {value!r}"
+ )
+ return value
class QueryConfigModel(_StrictModel):
@@ -240,6 +328,8 @@ def validate_rule_pack_payload(
__all__ = [
+ "XY_LAYOUT_CHART_TYPES",
+ "XY_STACKABLE_CHART_TYPES",
"DashboardConfigModel",
"GrafanaRulePackModel",
"IndexRewriteRuleModel",
diff --git a/observability_migration/adapters/source/grafana/panels.py b/observability_migration/adapters/source/grafana/panels.py
index ad3ebbcd..7472aa30 100644
--- a/observability_migration/adapters/source/grafana/panels.py
+++ b/observability_migration/adapters/source/grafana/panels.py
@@ -64,6 +64,7 @@
apply_style_guide_layout,
)
+from .extension_schema import XY_LAYOUT_CHART_TYPES, XY_STACKABLE_CHART_TYPES
from .extract import _normalize_text_panel_content
from .links import build_links_panel, translate_dashboard_links
from .manifest import (
@@ -4006,7 +4007,8 @@ def metrics_query_index(datasource_index=None, esql_index=None) -> str:
def translate_panel(panel, datasource_index="metrics-*", esql_index=None, rule_pack=None, resolver=None,
- llm_endpoint="", llm_model="", llm_api_key="", metric_series_labels=None):
+ llm_endpoint="", llm_model="", llm_api_key="", metric_series_labels=None,
+ section_title=""):
"""Translate a single Grafana panel, fusing multiple targets when possible."""
rule_pack = _rule_pack_for_panel(rule_pack or RulePackConfig(), panel)
# Single metrics read target for native PROMQL and ES|QL (see metrics_query_index).
@@ -4116,9 +4118,12 @@ def translate_panel(panel, datasource_index="metrics-*", esql_index=None, rule_p
)
if rule_pack.panel_query_overrides and kibana_type:
- _title_lower = (title or "").lower()
- for _override in rule_pack.panel_query_overrides:
- if _title_lower == (_override.get("title_match") or "").lower():
+ _selected_override = _select_panel_pack_override(
+ rule_pack.panel_query_overrides,
+ title,
+ section_title=section_title,
+ )
+ for _override in ([_selected_override] if _selected_override else []):
_curated_query = (_override.get("esql_query") or "").strip()
_status = _override.get("status_override") or "migrated"
if _curated_query and query_index:
@@ -11193,48 +11198,265 @@ def _iter_leaf_panels(panels: list[dict]):
yield panel
-def _apply_panel_layout_overrides_recursively(panels: list[dict], overrides: list[dict]) -> None:
+def _clear_duplicate_inner_title_label(panel: dict) -> None:
+ """Drop an inner metric/gauge label that merely repeats the chrome title."""
+ title = str(panel.get("title") or "").strip()
+ esql = panel.get("esql")
+ if not title or not isinstance(esql, dict):
+ return
+ chart_type = str(esql.get("type") or "")
+ if chart_type == "metric":
+ primary = esql.get("primary")
+ if isinstance(primary, dict):
+ label = str(primary.get("label") or "").strip()
+ if label.casefold() in {title.casefold(), "value", "computed_value"}:
+ # Blank, not omitted: Lens falls back to the field name
+ # (``value`` / ``computed_value``) when the label key is missing.
+ primary["label"] = " "
+ elif chart_type == "gauge":
+ metric = esql.get("metric")
+ if isinstance(metric, dict):
+ label = str(metric.get("label") or "").strip()
+ if label.casefold() in {title.casefold(), "value", "computed_value"}:
+ metric["label"] = " "
+
+
+def _pack_override_title_matches(title: str, override: dict) -> bool:
+ title_key = str(title or "").strip().casefold()
+ match = str(override.get("title_match") or "").strip().casefold()
+ return bool(match) and title_key == match
+
+
+def _pack_override_section_matches(section_title: str, override: dict) -> bool:
+ section_match = str(override.get("section_match") or "").strip().casefold()
+ if not section_match:
+ return True
+ current = str(section_title or "").strip().casefold()
+ return current == section_match or current.startswith(section_match)
+
+
+def _select_panel_pack_override(
+ overrides: list[dict], title: str, *, section_title: str = ""
+) -> dict | None:
+ """Pick a pack override for *title*, preferring a matching ``section_match``."""
+ generic = None
+ specific = None
+ for override in overrides or []:
+ if not _pack_override_title_matches(title, override):
+ continue
+ if str(override.get("section_match") or "").strip():
+ if _pack_override_section_matches(section_title, override):
+ specific = override
+ elif generic is None:
+ generic = override
+ return specific or generic
+
+
+def _layout_override_matches(
+ panel: dict, override: dict, *, section_title: str
+) -> bool:
+ if not _pack_override_title_matches(str(panel.get("title") or ""), override):
+ return False
+ return _pack_override_section_matches(section_title, override)
+
+
+def _layout_presentation_request(override: dict) -> list[str]:
+ """Names of the presentation keys *override* actually asks for."""
+ requested = []
+ for key in ("kibana_type_override", "xy_mode", "legend_position"):
+ value = override.get(key)
+ if isinstance(value, str) and value.strip():
+ requested.append(key)
+ return requested
+
+
+def _apply_layout_presentation_override(
+ panel: dict, override: dict, warnings: list | None
+) -> None:
+ """Apply the presentation-only part of a layout override (chart type,
+ stacking mode, legend placement).
+
+ This pass runs after translation and rewrites nothing but ``esql.type`` /
+ ``mode`` / ``legend``, so it can only move a panel *within* the XY family
+ (``line`` / ``bar`` / ``area``), which shares one schema shape (``query`` +
+ ``metrics``). A metric/gauge/datatable/pie panel has different required keys
+ (``primary`` / ``metric`` / ``breakdowns``) that only a new query can
+ produce, and every ``ESQL*PanelConfig`` in ``docs/dashboards/schema.json``
+ declares ``additionalProperties: false`` -- so stamping ``mode``/``legend``
+ onto a metric tile, or flipping an XY panel's ``type`` to ``metric``, emits
+ dashboard JSON the schema rejects. Those requests are skipped and reported
+ (``warnings``) instead of silently emitted or silently ignored;
+ ``panel.query_overrides`` is where an output-shape change belongs.
+ """
+ requested = _layout_presentation_request(override)
+ if not requested:
+ return
+ if isinstance(panel.get("section"), dict):
+ # A row container has no chart to present. Nothing to skip and no
+ # operator-visible gap: aiming a presentation key at a row title is a
+ # pack-authoring mistake, caught by the pack's own contract tests.
+ return
+
+ title = str(override.get("title_match") or panel.get("title") or "").strip()
+
+ def _warn(message: str) -> None:
+ if warnings is not None:
+ warnings.append((panel, message))
+
+ esql = panel.get("esql")
+ if not isinstance(esql, dict):
+ _warn(
+ f"curated layout override for panel '{title}' requested "
+ f"{', '.join(requested)}, but the migrated panel is not an ES|QL chart, "
+ "so the presentation change was skipped"
+ )
+ return
+
+ # An omitted ``type`` is a line chart (the schema's default), which is what
+ # the XY definitions assume.
+ current_type = str(esql.get("type") or "line").strip() or "line"
+ if current_type not in XY_LAYOUT_CHART_TYPES:
+ _warn(
+ f"curated layout override for panel '{title}' requested "
+ f"{', '.join(requested)}, but the migrated panel is a Kibana "
+ f"'{current_type}' chart: those keys only apply to the XY family "
+ f"{list(XY_LAYOUT_CHART_TYPES)}, so the presentation change was skipped. "
+ "Use panel.query_overrides to emit a different panel shape"
+ )
+ return
+
+ kibana_type = str(override.get("kibana_type_override") or "").strip()
+ type_changed = False
+ if kibana_type:
+ if kibana_type in XY_LAYOUT_CHART_TYPES:
+ type_changed = kibana_type != current_type
+ esql["type"] = kibana_type
+ else:
+ _warn(
+ f"curated layout override for panel '{title}' requested chart type "
+ f"'{kibana_type}', which is not one of the presentation-only XY types "
+ f"{list(XY_LAYOUT_CHART_TYPES)}; the panel kept its '{current_type}' "
+ "chart. Use panel.query_overrides to emit a different panel shape"
+ )
+ chart_type = str(esql.get("type") or "line").strip() or "line"
+
+ xy_mode = str(override.get("xy_mode") or "").strip()
+ if xy_mode and chart_type in XY_STACKABLE_CHART_TYPES:
+ esql["mode"] = xy_mode
+ elif xy_mode:
+ # ``line`` carries no stacking mode in the schema; dropping it keeps the
+ # emitted panel valid, but the request must still be visible.
+ esql.pop("mode", None)
+ _warn(
+ f"curated layout override for panel '{title}' requested xy_mode "
+ f"'{xy_mode}', which a Kibana '{chart_type}' chart does not support; the "
+ f"stacking request was dropped. Set kibana_type_override to one of "
+ f"{list(XY_STACKABLE_CHART_TYPES)} to stack it"
+ )
+ elif chart_type not in XY_STACKABLE_CHART_TYPES:
+ esql.pop("mode", None)
+ elif type_changed:
+ esql.setdefault("mode", "stacked")
+
+ legend_position = str(override.get("legend_position") or "").strip()
+ if legend_position:
+ legend = dict(esql.get("legend") or {})
+ legend["position"] = legend_position
+ legend.setdefault("visible", "show")
+ esql["legend"] = legend
+
+
+def _apply_one_panel_layout_override(
+ panel: dict, override: dict, warnings: list | None = None
+) -> None:
+ position_override = override.get("position") or {}
+ if position_override:
+ position = dict(panel.get("position", {}))
+ for key in ("x", "y"):
+ value = position_override.get(key)
+ if value is not None:
+ position[key] = int(value)
+ panel["position"] = position
+ size_override = override.get("size") or {}
+ if size_override:
+ size = dict(panel.get("size", {}))
+ for key in ("w", "h"):
+ value = size_override.get(key)
+ if value is not None:
+ size[key] = int(value)
+ panel["size"] = size
+ new_title = override.get("title")
+ if isinstance(new_title, str) and new_title.strip():
+ panel["title"] = new_title.strip()
+ if "collapsed" in override and isinstance(panel.get("section"), dict):
+ panel["section"]["collapsed"] = bool(override.get("collapsed"))
+ if "hide_title" in override and override.get("hide_title") is not None:
+ if override.get("hide_title"):
+ panel["hide_title"] = True
+ else:
+ panel.pop("hide_title", None)
+ _clear_duplicate_inner_title_label(panel)
+ _apply_layout_presentation_override(panel, override, warnings)
+
+
+def _apply_panel_layout_overrides_recursively(
+ panels: list[dict],
+ overrides: list[dict],
+ *,
+ section_title: str = "",
+ warnings: list | None = None,
+) -> None:
if not panels or not overrides:
return
- override_map = {
- str(override.get("title_match") or "").strip().casefold(): override
+ usable = [
+ override
for override in overrides
if str(override.get("title_match") or "").strip()
- }
- if not override_map:
+ ]
+ if not usable:
return
for panel in panels:
- title_key = str(panel.get("title") or "").strip().casefold()
- override = override_map.get(title_key)
- if override:
- position_override = override.get("position") or {}
- if position_override:
- position = dict(panel.get("position", {}))
- for key in ("x", "y"):
- value = position_override.get(key)
- if value is not None:
- position[key] = int(value)
- panel["position"] = position
- size_override = override.get("size") or {}
- if size_override:
- size = dict(panel.get("size", {}))
- for key in ("w", "h"):
- value = size_override.get(key)
- if value is not None:
- size[key] = int(value)
- panel["size"] = size
- new_title = override.get("title")
- if isinstance(new_title, str) and new_title.strip():
- panel["title"] = new_title.strip()
- if "collapsed" in override and isinstance(panel.get("section"), dict):
- panel["section"]["collapsed"] = bool(override.get("collapsed"))
+ # Child section_match is the Grafana row title, not a renamed Kibana title.
+ source_title = str(panel.get("title") or "")
+ for override in usable:
+ if _layout_override_matches(panel, override, section_title=section_title):
+ _apply_one_panel_layout_override(panel, override, warnings)
section = panel.get("section")
if isinstance(section, dict):
inner = section.get("panels")
if isinstance(inner, list):
- _apply_panel_layout_overrides_recursively(inner, overrides)
+ _apply_panel_layout_overrides_recursively(
+ inner,
+ overrides,
+ section_title=source_title,
+ warnings=warnings,
+ )
+
+
+def _attach_layout_override_warnings(result, yaml_doc: dict, warnings: list) -> None:
+ """Report skipped layout presentation overrides on the panels they targeted.
+
+ ``warnings`` holds ``(panel_dict, message)`` pairs collected while applying
+ the overrides; the panel dicts are the same objects still in ``yaml_doc``, so
+ leaf order lines up with ``result.yaml_panel_results`` (the same pairing
+ ``_sync_visual_ir`` uses below). A skipped presentation request caps the
+ panel at ``migrated_with_warnings``: the panel still renders, but it does not
+ look the way the pack asked for, and that is a gap the report must show.
+ """
+ by_panel: dict[int, list[str]] = {}
+ for panel, message in warnings:
+ by_panel.setdefault(id(panel), []).append(message)
+ for dashboard in yaml_doc.get("dashboards") or []:
+ leaves = list(_iter_leaf_panels(dashboard.get("panels") or []))
+ for panel, panel_result in zip(leaves, result.yaml_panel_results):
+ for message in by_panel.get(id(panel)) or []:
+ _append_unique(panel_result.reasons, message)
+ if panel_result.status == "migrated":
+ panel_result.status = "migrated_with_warnings"
+ panel_result.confidence = min(panel_result.confidence, 0.6)
+ recompute_result_counts(result)
def _resolve_section_overlaps_recursively(panels: list[dict]) -> None:
@@ -11312,6 +11534,7 @@ def _translate_panel_group(
llm_model="",
llm_api_key="",
metric_series_labels=None,
+ section_title="",
):
"""Translate a group of Grafana panels, returning (yaml_panels, panel_results)."""
yaml_panels: list[dict] = []
@@ -11333,6 +11556,7 @@ def _translate_panel_group(
llm_model=llm_model,
llm_api_key=llm_api_key,
metric_series_labels=metric_series_labels,
+ section_title=section_title,
)
result.panel_results.append(panel_result)
panel_result.operational_ir = build_operational_ir(
@@ -11505,6 +11729,7 @@ def translate_dashboard(dashboard, datasource_index="metrics-*", esql_index=None
llm_model=llm_model,
llm_api_key=llm_api_key,
metric_series_labels=metric_series_labels,
+ section_title=normalized_group.title or row_title or "",
)
result.yaml_panel_results.extend(panel_results)
@@ -11684,11 +11909,18 @@ def translate_dashboard(dashboard, datasource_index="metrics-*", esql_index=None
yaml_doc["dashboards"][0]["controls"] = controls
apply_style_guide_layout(yaml_doc)
+ # A layout override whose presentation request does not fit the translated
+ # panel shape is skipped rather than emitted as schema-invalid JSON; the
+ # skip is reported on the panel it was aimed at (never silently dropped).
+ layout_override_warnings: list = []
for dashboard in yaml_doc.get("dashboards") or []:
_apply_panel_layout_overrides_recursively(
dashboard.get("panels") or [],
getattr(rule_pack, "panel_layout_overrides", []) or [],
+ warnings=layout_override_warnings,
)
+ if layout_override_warnings:
+ _attach_layout_override_warnings(result, yaml_doc, layout_override_warnings)
# Safety net: ``apply_style_guide_layout`` (specifically
# ``_fill_simple_row``) can rescale a row's widths to total
diff --git a/observability_migration/adapters/source/grafana/rules.py b/observability_migration/adapters/source/grafana/rules.py
index 51391bd4..6deaf72c 100644
--- a/observability_migration/adapters/source/grafana/rules.py
+++ b/observability_migration/adapters/source/grafana/rules.py
@@ -291,6 +291,8 @@ def load_rule_pack_files(paths: Sequence[str] | None) -> RulePackConfig:
"esql_query": override.esql_query,
"status_override": override.status_override,
}
+ if override.section_match:
+ entry["section_match"] = override.section_match
if override.kibana_type_override:
entry["kibana_type_override"] = override.kibana_type_override
if override.drop_time_from:
@@ -323,6 +325,16 @@ def load_rule_pack_files(paths: Sequence[str] | None) -> RulePackConfig:
}
if override.title:
entry["title"] = override.title
+ if override.section_match:
+ entry["section_match"] = override.section_match
+ if override.hide_title is not None:
+ entry["hide_title"] = override.hide_title
+ if override.kibana_type_override:
+ entry["kibana_type_override"] = override.kibana_type_override
+ if override.xy_mode:
+ entry["xy_mode"] = override.xy_mode
+ if override.legend_position:
+ entry["legend_position"] = override.legend_position
pack.panel_layout_overrides.append(entry)
for field_name in (
@@ -398,6 +410,14 @@ def _load_curated_pack_for(dashboard: dict[str, Any]) -> RulePackConfig | None:
return pack
+def _override_layer_key(entry: dict) -> tuple[str, str]:
+ """Merge key for query/layout overrides: stripped, casefolded title + section."""
+ return (
+ str(entry.get("title_match") or "").strip().casefold(),
+ str(entry.get("section_match") or "").strip().casefold(),
+ )
+
+
def _merge_curated_into_base(curated: RulePackConfig, user: RulePackConfig) -> RulePackConfig:
"""Build a composed pack: curated as the base layer, user pack wins on collision."""
import copy
@@ -426,19 +446,20 @@ def _merge_curated_into_base(curated: RulePackConfig, user: RulePackConfig) -> R
result.panel_type_overrides.update(user.panel_type_overrides)
result.control_field_overrides.update(user.control_field_overrides)
- # panel_query_overrides: user overrides win by title_match
- user_override_titles = {o["title_match"] for o in user.panel_query_overrides}
+ # panel_query_overrides / panel_layout_overrides: user wins by
+ # (title_match, section_match), matching the runtime matcher which
+ # strips whitespace before casefolding.
+ user_query_keys = {_override_layer_key(o) for o in user.panel_query_overrides}
result.panel_query_overrides = [
o for o in result.panel_query_overrides
- if o["title_match"] not in user_override_titles
+ if _override_layer_key(o) not in user_query_keys
]
result.panel_query_overrides.extend(user.panel_query_overrides)
- # panel_layout_overrides: user overrides win by title_match
- user_layout_titles = {o["title_match"] for o in user.panel_layout_overrides}
+ user_layout_keys = {_override_layer_key(o) for o in user.panel_layout_overrides}
result.panel_layout_overrides = [
o for o in result.panel_layout_overrides
- if o["title_match"] not in user_layout_titles
+ if _override_layer_key(o) not in user_layout_keys
]
result.panel_layout_overrides.extend(user.panel_layout_overrides)
diff --git a/parity-rig/curated/grafana_763_redis_exporter/docker-compose.yml b/parity-rig/curated/grafana_763_redis_exporter/docker-compose.yml
index 90daef24..bfa31417 100644
--- a/parity-rig/curated/grafana_763_redis_exporter/docker-compose.yml
+++ b/parity-rig/curated/grafana_763_redis_exporter/docker-compose.yml
@@ -301,6 +301,7 @@ services:
command:
- |
until pg_isready -h postgres -U postgres >/dev/null 2>&1; do sleep 2; done
+ psql -h postgres -U postgres -d rigdb -c "CREATE EXTENSION IF NOT EXISTS pg_stat_statements;"
psql -h postgres -U postgres -d rigdb -c "
CREATE TABLE IF NOT EXISTS rig_load (
id SERIAL PRIMARY KEY,
@@ -328,6 +329,18 @@ services:
environment:
- POSTGRES_PASSWORD=rigpass
- POSTGRES_DB=rigdb
+ # Preload pg_stat_statements so the exporter's stat_statements collector has
+ # a view to read. Grafana 12485 "Query rate" / "Average query runtime" panels
+ # depend on pg_stat_statements_* series; without the extension they are an
+ # honest data gap rather than a rendered panel.
+ command:
+ - "postgres"
+ - "-c"
+ - "shared_preload_libraries=pg_stat_statements"
+ - "-c"
+ - "pg_stat_statements.track=all"
+ - "-c"
+ - "pg_stat_statements.max=10000"
networks:
- redis-rig
healthcheck:
@@ -341,6 +354,12 @@ services:
container_name: redis-rig-postgres-exporter
environment:
- DATA_SOURCE_NAME=postgresql://postgres:rigpass@postgres:5432/rigdb?sslmode=disable
+ # stat_statements: Query rate / Average query runtime panels (Grafana 12485).
+ # postmaster: pg_postmaster_start_time_seconds for the PostgreSQL Uptime panel.
+ # Both collectors are off by default in postgres_exporter v0.15.0.
+ command:
+ - "--collector.stat_statements"
+ - "--collector.postmaster"
ports:
- "9187:9187"
networks:
diff --git a/tests/test_curated_packs.py b/tests/test_curated_packs.py
index 69972cc3..7d785d08 100644
--- a/tests/test_curated_packs.py
+++ b/tests/test_curated_packs.py
@@ -7,10 +7,15 @@
import tomllib
from pathlib import Path
+import pytest
+
from observability_migration.adapters.source.grafana.curated_packs import (
find_curated_pack,
load_curated_registry,
)
+from observability_migration.adapters.source.grafana.extension_schema import (
+ validate_rule_pack_payload,
+)
from observability_migration.adapters.source.grafana.panels import (
_apply_panel_layout_overrides_recursively,
_label_placeholder_value_metric,
@@ -36,6 +41,28 @@
)
from observability_migration.adapters.source.grafana.schema import SchemaResolver
+DASHBOARD_SCHEMA_PATH = (
+ Path(__file__).resolve().parents[1] / "docs" / "dashboards" / "schema.json"
+)
+
+
+def dashboard_schema_errors(panels: list[dict]) -> list[str]:
+ """Validate *panels* as a dashboard against the vendored Kibana schema.
+
+ Same schema and validator as the ``tests/e2e`` schema gate, applied to a
+ single hand-built dashboard so a layout-override regression is caught in the
+ fast unit gate instead of only after a corpus run.
+ """
+ import jsonschema
+
+ schema = json.loads(DASHBOARD_SCHEMA_PATH.read_text())
+ doc = {"dashboards": [{"name": "layout-override-probe", "panels": panels}]}
+ return [
+ f"{'/'.join(str(part) for part in error.path)}: {error.message}"
+ for error in jsonschema.Draft202012Validator(schema).iter_errors(doc)
+ ]
+
+
# ---------------------------------------------------------------------------
# Registry loader
# ---------------------------------------------------------------------------
@@ -153,7 +180,7 @@ def test_registry_pins_match_community_corpus_when_revision_aligns():
for entry in corpus["dashboards"]
}
# New packs in this PR. 9628 is pack rev 1 vs corpus rev 8 — no join.
- new_pack_ids = {7362, 9628, 14114}
+ new_pack_ids = {7362, 9628, 14114, 12485}
mismatches = []
for entry in load_curated_registry():
gnet_id = int(entry["gnet_id"])
@@ -1787,6 +1814,549 @@ def test_14114_instance_up_becomes_pg_up_control():
assert "metrics.up" not in query
+def test_find_12485_by_gnet_id():
+ entry = find_curated_pack(gnet_id=12485, title="", tags=[])
+ assert entry is not None
+ assert entry["gnet_id"] == 12485
+ assert entry["name"] == "grafana_12485_postgresql_exporter"
+
+
+def test_resolve_pack_12485_pins_kinds_renames_and_controls():
+ dashboard = {"gnetId": 12485, "title": "PostgreSQL Exporter", "tags": []}
+ resolved = resolve_pack_for_dashboard(dashboard, RulePackConfig())
+ # The two `_count` gauges are the important correctness fix.
+ assert resolved.metric_kinds.get("pg_stat_activity_count") == "gauge"
+ assert resolved.metric_kinds.get("pg_locks_count") == "gauge"
+ assert resolved.metric_kinds.get("pg_stat_database_numbackends") == "gauge"
+ # Rated counters stay counters.
+ assert resolved.metric_kinds.get("pg_stat_database_xact_commit") == "counter"
+ assert resolved.metric_kinds.get("pg_stat_database_tup_fetched") == "counter"
+ # v0.15 renames.
+ for src, tgt in (
+ ("pg_database_size", "metrics.pg_database_size_bytes"),
+ ("pg_replication_lag", "metrics.pg_replication_lag_seconds"),
+ ("pg_stat_statements_calls", "metrics.pg_stat_statements_calls_total"),
+ ("pg_stat_statements_total_time_seconds", "metrics.pg_stat_statements_seconds_total"),
+ ):
+ entry = (resolved.metric_map or {}).get(src)
+ assert getattr(entry, "target", entry) == tgt, src
+ # Controls keyed by the dashboard's capitalised variable names.
+ assert resolved.control_field_overrides.get("Instance") == "labels.instance"
+ assert resolved.control_field_overrides.get("Database") == "labels.datname"
+
+
+def test_12485_database_size_renamed_offline():
+ dashboard = {"gnetId": 12485, "title": "PostgreSQL Exporter", "tags": []}
+ resolved = resolve_pack_for_dashboard(dashboard, RulePackConfig())
+ resolver = SchemaResolver(resolved)
+ panel = {
+ "id": 37,
+ "type": "singlestat",
+ "title": "Total database size",
+ "targets": [{"expr": 'sum(pg_database_size{instance="$Instance"})', "refId": "A"}],
+ "gridPos": {"x": 0, "y": 0, "w": 4, "h": 3},
+ }
+ yaml_panel, result = translate_panel(
+ panel,
+ datasource_index="metrics-*",
+ esql_index="metrics-*",
+ rule_pack=resolved,
+ resolver=resolver,
+ )
+ assert result.status in {"migrated", "migrated_with_warnings"}, result.reasons
+ query = (yaml_panel.get("esql") or {}).get("query") or ""
+ assert "metrics.pg_database_size_bytes" in query
+ assert "metrics.pg_database_size)" not in query
+
+
+def test_12485_activity_count_is_gauge_not_rated():
+ dashboard = {"gnetId": 12485, "title": "PostgreSQL Exporter", "tags": []}
+ resolved = resolve_pack_for_dashboard(dashboard, RulePackConfig())
+ resolver = SchemaResolver(resolved)
+ panel = {
+ "id": 24,
+ "type": "graph",
+ "title": "Connections by state (stacked)",
+ "targets": [{"expr": 'sum by (state) (pg_stat_activity_count{instance="$Instance"})', "refId": "A"}],
+ "gridPos": {"x": 0, "y": 0, "w": 12, "h": 7},
+ }
+ yaml_panel, result = translate_panel(
+ panel,
+ datasource_index="metrics-*",
+ esql_index="metrics-*",
+ rule_pack=resolved,
+ resolver=resolver,
+ )
+ assert result.status in {"migrated", "migrated_with_warnings"}, result.reasons
+ query = (yaml_panel.get("esql") or {}).get("query") or ""
+ assert "pg_stat_activity_count" in query
+ # gauge → SUM, never RATE/IRATE (the whole point of forcing the _count gauge).
+ assert "RATE(" not in query.upper()
+ assert "labels.state" in query
+
+
+def test_12485_instance_and_database_controls_rewritten():
+ dashboard = {
+ "gnetId": 12485,
+ "title": "PostgreSQL Exporter",
+ "tags": [],
+ "templating": {
+ "list": [
+ {
+ "name": "Instance",
+ "type": "query",
+ "label": "Instance",
+ "query": 'label_values({job="postgres-exporter"}, instance)',
+ "includeAll": False,
+ "current": {"text": "postgres:5432", "value": "postgres:5432"},
+ },
+ {
+ "name": "Database",
+ "type": "query",
+ "label": "Database",
+ "query": "label_values(datname)",
+ "includeAll": True,
+ "current": {"text": "All", "value": "$__all"},
+ },
+ {
+ "name": "Interval",
+ "type": "interval",
+ "query": "30sec,1m,10m,30m,1h,6h,12h,1d",
+ "current": {"text": "1m", "value": "1m"},
+ },
+ ]
+ },
+ "panels": [
+ {
+ "id": 14,
+ "type": "singlestat",
+ "title": "Transaction rate",
+ "targets": [
+ {
+ "expr": 'sum(rate(pg_stat_database_xact_commit{instance="$Instance",datname=~"$Database"}[$Interval]))',
+ "refId": "A",
+ }
+ ],
+ "gridPos": {"x": 0, "y": 0, "w": 4, "h": 3},
+ }
+ ],
+ }
+ resolved = resolve_pack_for_dashboard(dashboard, RulePackConfig())
+ result = translate_dashboard(
+ dashboard,
+ datasource_index="metrics-*",
+ esql_index="metrics-*",
+ rule_pack=resolved,
+ )
+ payload = result.dashboard_ir.to_yaml_dict()
+ controls = payload.get("controls") or []
+ names = {c.get("variable_name") for c in controls}
+ # Interval must never become a control; Instance/Database must.
+ assert "Interval" not in names, controls
+ assert "Instance" in names, controls
+ instance = next(c for c in controls if c.get("variable_name") == "Instance")
+ iq = str(instance.get("query") or "")
+ assert "pg_up" in iq
+ assert "postgres-exporter" not in iq
+ # Database is a core claim of the pack: the bare ``label_values(datname)``
+ # has no metric anchor, so the control must be present AND anchored on the
+ # curated per-database gauge (an unanchored ``labels.datname`` query is the
+ # broken pre-pack behavior, not an acceptable fallback).
+ assert "Database" in names, controls
+ database = next(c for c in controls if c.get("variable_name") == "Database")
+ dq = str(database.get("query") or "")
+ assert "pg_stat_database_numbackends" in dq, dq
+
+
+def _pinned_12485_repeat_row_dashboard() -> dict:
+ """Minimal dashboard in the shape of pinned grafana.com 12485 revision 1.
+
+ Faithful to the parts this test is about: ``Database`` is a
+ ``multi``/``includeAll`` query variable with no cached ``current``/
+ ``options``, and the ``Database: $Database`` row is a *collapsed repeated*
+ row (``repeat: Database``) holding the per-database panels.
+ """
+ return {
+ "gnetId": 12485,
+ "title": "PostgreSQL Exporter",
+ "tags": [],
+ "templating": {
+ "list": [
+ {
+ "name": "Instance",
+ "type": "query",
+ "label": "Instance",
+ "query": 'label_values({job="postgres-exporter"}, instance)',
+ "includeAll": False,
+ "multi": False,
+ "current": {},
+ "options": [],
+ },
+ {
+ "name": "Database",
+ "type": "query",
+ "label": "Database",
+ "query": "label_values(datname)",
+ "includeAll": True,
+ "multi": True,
+ "current": {},
+ "options": [],
+ },
+ ]
+ },
+ "panels": [
+ {
+ "id": 2,
+ "type": "row",
+ "title": "Global Statistics",
+ "collapsed": False,
+ "panels": [],
+ "gridPos": {"x": 0, "y": 0, "w": 24, "h": 1},
+ },
+ {
+ "id": 14,
+ "type": "singlestat",
+ "title": "Transaction rate",
+ "targets": [
+ {
+ "expr": 'sum(rate(pg_stat_database_xact_commit{instance="$Instance"}[5m]))',
+ "refId": "A",
+ }
+ ],
+ "gridPos": {"x": 0, "y": 1, "w": 4, "h": 3},
+ },
+ {
+ "id": 100,
+ "type": "row",
+ "title": "Database: $Database",
+ "repeat": "Database",
+ "collapsed": True,
+ "gridPos": {"x": 0, "y": 10, "w": 24, "h": 1},
+ "panels": [
+ {
+ "id": 101,
+ "type": "singlestat",
+ "title": "Active clients",
+ "targets": [
+ {
+ "expr": (
+ 'sum(pg_stat_activity_count{instance="$Instance",'
+ 'datname=~"$Database",state="active"})'
+ ),
+ "refId": "A",
+ }
+ ],
+ "gridPos": {"x": 0, "y": 11, "w": 4, "h": 3},
+ }
+ ],
+ },
+ ],
+ }
+
+
+def test_12485_repeated_database_row_becomes_single_select_control():
+ """The pinned source repeats the Database row over a multi-select variable.
+
+ Kibana cannot repeat panels, so the engine deliberately emits ONE section
+ with a single-select Database control plus an explicit warning. The pack's
+ fidelity manifest has to disclose that (see the manifest test below), and
+ this test pins the behavior the disclosure describes.
+ """
+ dashboard = _pinned_12485_repeat_row_dashboard()
+ resolved = resolve_pack_for_dashboard(dashboard, RulePackConfig())
+
+ result = translate_dashboard(
+ dashboard,
+ datasource_index="metrics-*",
+ esql_index="metrics-*",
+ rule_pack=resolved,
+ )
+
+ controls = result.dashboard_ir.to_yaml_dict().get("controls") or []
+ database = next(
+ (c for c in controls if c.get("variable_name") == "Database"), None
+ )
+ assert database is not None, controls
+ assert database.get("multiple") is False, database
+ assert any(
+ "drives panel repetition" in warning for warning in result.control_warnings
+ ), result.control_warnings
+
+
+def test_12485_fidelity_manifest_discloses_repeated_database_row_gap():
+ """Repo rule: an operator-visible structural loss must be disclosed.
+
+ 28 PERFECT panel labels and a repopulated Database control must not read as
+ "Grafana's repeated per-database rows were preserved" -- they were not.
+ """
+ import yaml
+
+ from observability_migration.adapters.source.grafana import (
+ curated_packs as _curated_packs_pkg,
+ )
+
+ manifest_path = (
+ Path(_curated_packs_pkg.__file__).parent
+ / "grafana_12485_postgresql_exporter"
+ / "fidelity_manifest.yaml"
+ )
+ manifest = yaml.safe_load(manifest_path.read_text()) or {}
+ known_gaps = [
+ str(gap) for gap in (manifest.get("summary") or {}).get("known_gaps") or []
+ ]
+
+ disclosure = [
+ gap
+ for gap in known_gaps
+ if "repeat" in gap.lower() and "single-select" in gap.lower()
+ ]
+ assert disclosure, (
+ "fidelity_manifest.yaml must disclose that the repeated 'Database: "
+ f"$Database' row becomes one single-select section; known_gaps={known_gaps}"
+ )
+ assert "Database" in disclosure[0]
+
+
+def test_12485_io_override_names_read_and_write():
+ dashboard = {"gnetId": 12485, "title": "PostgreSQL Exporter", "tags": []}
+ resolved = resolve_pack_for_dashboard(dashboard, RulePackConfig())
+ resolver = SchemaResolver(resolved)
+ panel = {
+ "id": 26,
+ "type": "graph",
+ "title": "I/O Read/Write time",
+ "targets": [
+ {
+ "expr": 'sum(rate(pg_stat_database_blk_read_time{instance="$Instance"}[1m]))',
+ "legendFormat": "blk_read_time",
+ "refId": "A",
+ },
+ {
+ "expr": 'sum(rate(pg_stat_database_blk_write_time{instance="$Instance"}[1m]))',
+ "legendFormat": "blk_read_time",
+ "refId": "B",
+ },
+ ],
+ "gridPos": {"x": 0, "y": 0, "w": 12, "h": 9},
+ }
+ yaml_panel, result = translate_panel(
+ panel,
+ datasource_index="metrics-*",
+ esql_index="metrics-*",
+ rule_pack=resolved,
+ resolver=resolver,
+ )
+ assert result.status in {"migrated", "migrated_with_warnings"}, result.reasons
+ query = (yaml_panel.get("esql") or {}).get("query") or ""
+ assert "Read =" in query
+ assert "Write =" in query
+ assert "blk_read_time_B" not in query
+
+
+def test_12485_avg_query_runtime_skips_null_last_bucket():
+ dashboard = {"gnetId": 12485, "title": "PostgreSQL Exporter", "tags": []}
+ resolved = resolve_pack_for_dashboard(dashboard, RulePackConfig())
+ resolver = SchemaResolver(resolved)
+ panel = {
+ "id": 102,
+ "type": "singlestat",
+ "title": "Average query runtime",
+ "format": "s",
+ "valueName": "current",
+ "targets": [
+ {
+ "expr": (
+ 'sum((delta(pg_stat_statements_total_time_seconds'
+ '{instance="$Instance"}[5m])))'
+ '/sum((delta(pg_stat_statements_calls'
+ '{instance="$Instance"}[5m])))'
+ ),
+ "refId": "A",
+ },
+ ],
+ "gridPos": {"x": 8, "y": 7, "w": 4, "h": 3},
+ }
+ yaml_panel, result = translate_panel(
+ panel,
+ datasource_index="metrics-*",
+ esql_index="metrics-*",
+ rule_pack=resolved,
+ resolver=resolver,
+ )
+ assert result.status in {"migrated", "migrated_with_warnings"}, result.reasons
+ esql = yaml_panel.get("esql") or {}
+ query = esql.get("query") or ""
+ assert "LAST(value, step)" not in query
+ assert "computed_value" in query
+ assert "WHERE computed_value IS NOT NULL" in query
+ assert "LIMIT 2" in query
+ assert "RATE(" in query
+ assert "pg_stat_statements_seconds_total" in query
+ assert "pg_stat_statements_calls_total" in query
+ primary = esql.get("primary") or {}
+ assert (primary.get("format") or {}).get("type") == "duration"
+
+
+def test_12485_deadlocks_override_legends_by_datname_and_scopes_database():
+ dashboard = {"gnetId": 12485, "title": "PostgreSQL Exporter", "tags": []}
+ resolved = resolve_pack_for_dashboard(dashboard, RulePackConfig())
+ resolver = SchemaResolver(resolved)
+ panel = {
+ "id": 30,
+ "type": "graph",
+ "title": "Deadlocks by database",
+ "legend": {"show": True, "hideZero": True, "hideEmpty": True},
+ "targets": [
+ {
+ "expr": (
+ 'sum by (datname) ((rate(pg_stat_database_deadlocks'
+ '{instance="$Instance"}[5m])))'
+ ),
+ "legendFormat": "{{datname}}",
+ "refId": "A",
+ },
+ ],
+ "gridPos": {"x": 12, "y": 37, "w": 12, "h": 5},
+ }
+
+ global_yaml, global_result = translate_panel(
+ panel,
+ datasource_index="metrics-*",
+ esql_index="metrics-*",
+ rule_pack=resolved,
+ resolver=resolver,
+ section_title="Global Statistics",
+ )
+ assert global_result.status in {"migrated", "migrated_with_warnings"}, global_result.reasons
+ global_query = (global_yaml.get("esql") or {}).get("query") or ""
+ assert "labels.datname" in global_query
+ assert "?Database" not in global_query
+ assert "LAST(value, step)" not in global_query
+
+ db_yaml, db_result = translate_panel(
+ panel,
+ datasource_index="metrics-*",
+ esql_index="metrics-*",
+ rule_pack=resolved,
+ resolver=resolver,
+ section_title="Database: $Database",
+ )
+ assert db_result.status in {"migrated", "migrated_with_warnings"}, db_result.reasons
+ db_query = (db_yaml.get("esql") or {}).get("query") or ""
+ assert "labels.datname" in db_query
+ assert "?Database" in db_query
+
+
+def test_panel_layout_overrides_can_move_legend_right():
+ panels = [
+ {
+ "title": "Locks by state",
+ "esql": {
+ "type": "bar",
+ "mode": "stacked",
+ "query": "FROM metrics-*",
+ "legend": {"visible": "show", "position": "bottom"},
+ },
+ "position": {"x": 0, "y": 0},
+ "size": {"w": 24, "h": 16},
+ }
+ ]
+ overrides = [
+ {
+ "title_match": "Locks by state",
+ "legend_position": "right",
+ }
+ ]
+
+ _apply_panel_layout_overrides_recursively(panels, overrides)
+
+ assert panels[0]["esql"]["legend"]["position"] == "right"
+
+
+def test_12485_layout_fills_kpi_row_and_unhides_gauges():
+ dashboard = {"gnetId": 12485, "title": "PostgreSQL Exporter", "tags": []}
+ resolved = resolve_pack_for_dashboard(dashboard, RulePackConfig())
+ by_title = {}
+ for item in resolved.panel_layout_overrides:
+ key = (item["title_match"], item.get("section_match") or "")
+ by_title[key] = item
+ assert by_title[("Total database size", "")]["size"]["w"] == 12
+ assert by_title[("Average query runtime", "")]["position"]["x"] == 12
+ assert by_title[("Shared Buffers", "")]["position"]["x"] == 24
+ assert by_title[("Max Connections", "")]["position"]["x"] == 36
+ assert "y" not in by_title[("Total database size", "")].get("position", {})
+ assert by_title[("Shared Buffer Hits", "")]["hide_title"] is False
+ assert by_title[("Connections used", "")]["hide_title"] is False
+ assert by_title[("Commit Ratio", "")]["hide_title"] is False
+ assert by_title[("Database", "")]["collapsed"] is False
+ assert by_title[("Locks by state", "")]["kibana_type_override"] == "bar"
+ assert by_title[("Locks by state", "")]["xy_mode"] == "stacked"
+ assert by_title[("Locks by state", "")]["legend_position"] == "right"
+ assert by_title[("Replication lag", "")]["size"]["w"] == 48
+ assert by_title[("Replication lag", "")]["position"]["y"] == 84
+ assert by_title[("I/O Read/Write time", "")]["size"]["h"] == 16
+ assert by_title[("Transactions", "Global Statistics")]["position"]["y"] == 36
+ assert by_title[("Active clients", "Database")]["position"] == {"x": 0, "y": 0}
+ assert by_title[("Transaction rate", "Database")]["size"]["w"] == 16
+ assert by_title[("Temporary files by database", "Database")]["position"]["y"] == 58
+
+
+def test_14114_numbackends_override_drops_name_breakdown():
+ dashboard = {
+ "gnetId": 14114,
+ "title": "PostgreSQL Exporter Quickstart and Dashboard",
+ "tags": ["postgres"],
+ }
+ resolved = resolve_pack_for_dashboard(dashboard, RulePackConfig())
+ resolver = SchemaResolver(resolved)
+ panel = {
+ "id": 6,
+ "type": "graph",
+ "title": "Number of active connections",
+ "targets": [
+ {
+ "expr": 'pg_stat_database_numbackends{datname=~"$db",instance=~"$instance"}',
+ "legendFormat": "{{__name__}}",
+ "refId": "A",
+ }
+ ],
+ "gridPos": {"x": 12, "y": 14, "w": 12, "h": 7},
+ }
+ yaml_panel, result = translate_panel(
+ panel,
+ datasource_index="metrics-*",
+ esql_index="metrics-*",
+ rule_pack=resolved,
+ resolver=resolver,
+ )
+ assert result.status in {"migrated", "migrated_with_warnings"}, result.reasons
+ esql = yaml_panel.get("esql") or {}
+ query = esql.get("query") or ""
+ assert "connections =" in query
+ assert "__name__" not in query
+ assert "GROK" not in query
+ assert "labels.datname" in query
+ # Source is one series per (instance, datname); grouping only by datname
+ # would MAX-collapse two exporters that share a database name.
+ assert "labels.instance" in query
+ stats_line = next(
+ line for line in query.splitlines() if "STATS connections" in line
+ )
+ assert "labels.instance" in stats_line
+ assert "labels.datname" in stats_line
+ assert (esql.get("breakdown") or {}).get("field") == "series_group"
+ assert "EVAL series_group = CONCAT(" in query
+ qps = next(
+ item for item in resolved.panel_layout_overrides if item["title_match"] == "QPS"
+ )
+ assert qps["size"]["h"] == 14
+ assert qps["size"]["w"] == 8
+ rows = next(
+ item for item in resolved.panel_layout_overrides if item["title_match"] == "Rows"
+ )
+ assert rows["size"] == {"w": 40, "h": 14}
+
+
def test_prometheus_native_label_candidates_come_first_in_redis_packs():
"""Offline runs take the FIRST candidate without probing the target.
@@ -1803,6 +2373,7 @@ def test_prometheus_native_label_candidates_come_first_in_redis_packs():
7362: [("instance", "labels.instance"), ("job", "labels.job")],
9628: [("instance", "labels.instance"), ("job", "labels.job")],
14114: [("instance", "labels.instance"), ("job", "labels.job")],
+ 12485: [("instance", "labels.instance"), ("job", "labels.job")],
}
for gnet_id, pairs in expected_first.items():
resolved = resolve_pack_for_dashboard(
@@ -2151,6 +2722,61 @@ def test_panel_layout_override_user_wins_over_curated():
assert merged.panel_layout_overrides == user.panel_layout_overrides
+def test_panel_layout_override_user_section_does_not_drop_other_section():
+ """A user override for one section must not delete the curated sibling."""
+ from observability_migration.adapters.source.grafana.rules import _merge_curated_into_base
+
+ curated = RulePackConfig()
+ curated.panel_layout_overrides = [
+ {"title_match": "Transactions", "section_match": "Global", "size": {"w": 24}},
+ {"title_match": "Transactions", "section_match": "Database", "size": {"w": 16}},
+ {"title_match": "Locks by state", "xy_mode": "stacked"},
+ ]
+ curated._curated_pack_name = "test_curated"
+
+ user = RulePackConfig()
+ user.panel_layout_overrides = [
+ {"title_match": "Transactions", "section_match": "Database", "size": {"w": 12}},
+ ]
+
+ merged = _merge_curated_into_base(curated, user)
+ by_key = {
+ (item["title_match"], item.get("section_match") or ""): item
+ for item in merged.panel_layout_overrides
+ }
+ assert by_key[("Transactions", "Global")]["size"] == {"w": 24}
+ assert by_key[("Transactions", "Database")]["size"] == {"w": 12}
+ assert by_key[("Locks by state", "")]["xy_mode"] == "stacked"
+
+
+def test_panel_override_merge_strips_whitespace_keys():
+ """Padded user keys must replace the matching curated override, not layer both."""
+ from observability_migration.adapters.source.grafana.rules import _merge_curated_into_base
+
+ curated = RulePackConfig()
+ curated.panel_query_overrides = [
+ {"title_match": "Transactions", "section_match": "Database", "esql_query": "-- curated"},
+ ]
+ curated.panel_layout_overrides = [
+ {"title_match": "Locks by state", "xy_mode": "stacked"},
+ ]
+ curated._curated_pack_name = "test_curated"
+
+ user = RulePackConfig()
+ user.panel_query_overrides = [
+ {"title_match": " Transactions ", "section_match": " Database ", "esql_query": "-- user"},
+ ]
+ user.panel_layout_overrides = [
+ {"title_match": " Locks by state ", "xy_mode": "grouped"},
+ ]
+
+ merged = _merge_curated_into_base(curated, user)
+ assert len(merged.panel_query_overrides) == 1
+ assert merged.panel_query_overrides[0]["esql_query"] == "-- user"
+ assert len(merged.panel_layout_overrides) == 1
+ assert merged.panel_layout_overrides[0]["xy_mode"] == "grouped"
+
+
def test_panel_layout_overrides_apply_inside_sections():
panels = [
{
@@ -2207,6 +2833,384 @@ def test_panel_layout_overrides_can_flip_section_collapsed_state():
assert panels[0]["section"]["collapsed"] is False
+def test_panel_layout_overrides_can_unhide_gauge_title():
+ panels = [
+ {
+ "title": "Shared Buffer Hits",
+ "hide_title": True,
+ "esql": {
+ "type": "gauge",
+ "metric": {"field": "value", "label": "Shared Buffer Hits"},
+ },
+ "position": {"x": 0, "y": 0},
+ "size": {"w": 8, "h": 14},
+ }
+ ]
+ overrides = [{"title_match": "Shared Buffer Hits", "hide_title": False}]
+
+ _apply_panel_layout_overrides_recursively(panels, overrides)
+
+ assert "hide_title" not in panels[0]
+ assert panels[0]["esql"]["metric"]["label"] == " "
+
+
+def test_panel_layout_overrides_section_match_skips_other_section():
+ panels = [
+ {
+ "title": "Global Statistics",
+ "section": {
+ "collapsed": False,
+ "panels": [
+ {
+ "title": "Transaction rate",
+ "position": {"x": 8, "y": 6},
+ "size": {"w": 8, "h": 5},
+ }
+ ],
+ },
+ },
+ {
+ "title": "Database",
+ "section": {
+ "collapsed": True,
+ "panels": [
+ {
+ "title": "Transaction rate",
+ "position": {"x": 0, "y": 5},
+ "size": {"w": 8, "h": 5},
+ }
+ ],
+ },
+ },
+ ]
+ overrides = [
+ {
+ "title_match": "Transaction rate",
+ "section_match": "Database",
+ "position": {"x": 32, "y": 0},
+ "size": {"w": 16, "h": 8},
+ }
+ ]
+
+ _apply_panel_layout_overrides_recursively(panels, overrides)
+
+ global_txn = panels[0]["section"]["panels"][0]
+ db_txn = panels[1]["section"]["panels"][0]
+ assert global_txn["position"] == {"x": 8, "y": 6}
+ assert db_txn["position"] == {"x": 32, "y": 0}
+ assert db_txn["size"] == {"w": 16, "h": 8}
+
+
+def test_panel_layout_overrides_can_force_stacked_bar():
+ panels = [
+ {
+ "title": "Locks by state",
+ "esql": {"type": "line", "query": "FROM metrics-*"},
+ "position": {"x": 0, "y": 0},
+ "size": {"w": 24, "h": 16},
+ }
+ ]
+ overrides = [
+ {
+ "title_match": "Locks by state",
+ "kibana_type_override": "bar",
+ "xy_mode": "stacked",
+ }
+ ]
+
+ _apply_panel_layout_overrides_recursively(panels, overrides)
+
+ assert panels[0]["esql"]["type"] == "bar"
+ assert panels[0]["esql"]["mode"] == "stacked"
+
+
+def test_panel_layout_overrides_xy_mode_without_type_override():
+ panels = [
+ {
+ "title": "Locks by state",
+ "esql": {"type": "bar", "query": "FROM metrics-*"},
+ "position": {"x": 0, "y": 0},
+ "size": {"w": 24, "h": 16},
+ }
+ ]
+ overrides = [
+ {
+ "title_match": "Locks by state",
+ "xy_mode": "stacked",
+ }
+ ]
+
+ _apply_panel_layout_overrides_recursively(panels, overrides)
+
+ assert panels[0]["esql"]["type"] == "bar"
+ assert panels[0]["esql"]["mode"] == "stacked"
+
+
+# ---------------------------------------------------------------------------
+# layout_overrides presentation contract (schema-valid or reported)
+# ---------------------------------------------------------------------------
+
+def _metric_probe_panel() -> dict:
+ return {
+ "title": "PostgreSQL Uptime",
+ "esql": {
+ "type": "metric",
+ "query": "FROM metrics-* | STATS value = MAX(uptime)",
+ "primary": {"field": "value", "label": "PostgreSQL Uptime"},
+ },
+ "position": {"x": 0, "y": 0},
+ "size": {"w": 12, "h": 8},
+ }
+
+
+def _xy_probe_panel(chart_type: str = "line") -> dict:
+ esql = {
+ "type": chart_type,
+ "query": (
+ "FROM metrics-* | STATS value = SUM(locks) BY "
+ "time_bucket = TBUCKET(1 hour), `labels.mode`"
+ ),
+ "dimension": {"field": "time_bucket"},
+ "metrics": [{"field": "value"}],
+ "breakdown": {"field": "labels.mode"},
+ }
+ if chart_type in {"bar", "area"}:
+ esql["mode"] = "stacked"
+ return {
+ "title": "Locks by state",
+ "esql": esql,
+ "position": {"x": 0, "y": 0},
+ "size": {"w": 24, "h": 16},
+ }
+
+
+def test_layout_override_presentation_keys_skip_non_xy_panel():
+ """``xy_mode``/``legend_position`` on a metric tile would add ``mode``/
+ ``legend`` keys that ``ESQLMetricPanelConfig`` (``additionalProperties:
+ false``) rejects. Skip and report instead of emitting invalid JSON."""
+ panels = [_metric_probe_panel()]
+ warnings: list = []
+
+ _apply_panel_layout_overrides_recursively(
+ panels,
+ [
+ {
+ "title_match": "PostgreSQL Uptime",
+ "xy_mode": "stacked",
+ "legend_position": "right",
+ }
+ ],
+ warnings=warnings,
+ )
+
+ esql = panels[0]["esql"]
+ assert esql["type"] == "metric"
+ assert "mode" not in esql
+ assert "legend" not in esql
+ assert warnings, "a skipped presentation request must be reported"
+ message = warnings[0][1]
+ assert "PostgreSQL Uptime" in message
+ assert "xy_mode" in message and "legend_position" in message
+ assert "query_overrides" in message
+ assert not dashboard_schema_errors(panels)
+
+
+def test_layout_override_cannot_change_panel_shape():
+ """A late ``type`` flip keeps the XY ``metrics``/``dimension`` columns and
+ has no ``primary``, so ``metric`` output would fail the schema both ways."""
+ panels = [_xy_probe_panel("line")]
+ warnings: list = []
+
+ _apply_panel_layout_overrides_recursively(
+ panels,
+ [{"title_match": "Locks by state", "kibana_type_override": "metric"}],
+ warnings=warnings,
+ )
+
+ assert panels[0]["esql"]["type"] == "line"
+ assert warnings
+ assert "query_overrides" in warnings[0][1]
+ assert not dashboard_schema_errors(panels)
+
+
+def test_layout_override_xy_mode_on_line_panel_is_reported():
+ """``ESQLLinePanelConfig`` has no ``mode``; dropping it keeps the panel
+ valid, but the ignored stacking request must still be visible."""
+ panels = [_xy_probe_panel("line")]
+ warnings: list = []
+
+ _apply_panel_layout_overrides_recursively(
+ panels,
+ [{"title_match": "Locks by state", "xy_mode": "stacked"}],
+ warnings=warnings,
+ )
+
+ assert panels[0]["esql"]["type"] == "line"
+ assert "mode" not in panels[0]["esql"]
+ assert warnings
+ assert "kibana_type_override" in warnings[0][1]
+ assert not dashboard_schema_errors(panels)
+
+
+def test_layout_override_presentation_output_is_schema_valid():
+ """Every presentation override shape -- including the ones that used to
+ emit invalid JSON -- validates against ``docs/dashboards/schema.json``."""
+ cases = [
+ (_metric_probe_panel(), {"xy_mode": "stacked"}),
+ (_metric_probe_panel(), {"legend_position": "right"}),
+ (_metric_probe_panel(), {"kibana_type_override": "gauge"}),
+ (_xy_probe_panel("line"), {"kibana_type_override": "metric"}),
+ (_xy_probe_panel("line"), {"kibana_type_override": "datatable"}),
+ (_xy_probe_panel("line"), {"xy_mode": "percentage"}),
+ (_xy_probe_panel("bar"), {"kibana_type_override": "line"}),
+ (_xy_probe_panel("bar"), {"xy_mode": "percentage"}),
+ # The real 12485 rule: composition-over-time line -> stacked bar with
+ # the lock-mode legend moved out from under the plot.
+ (
+ _xy_probe_panel("line"),
+ {
+ "kibana_type_override": "bar",
+ "xy_mode": "stacked",
+ "legend_position": "right",
+ },
+ ),
+ ]
+ failures = []
+ for panel, override in cases:
+ panels = [panel]
+ _apply_panel_layout_overrides_recursively(
+ panels, [{"title_match": panel["title"], **override}]
+ )
+ errors = dashboard_schema_errors(panels)
+ if errors:
+ failures.append(f"{override} -> {errors}")
+ assert not failures, "\n".join(failures)
+
+
+def test_12485_locks_layout_override_still_emits_stacked_bar():
+ """The pack's own line -> stacked bar + right legend must keep working."""
+ dashboard = {"gnetId": 12485, "title": "PostgreSQL Exporter", "tags": []}
+ resolved = resolve_pack_for_dashboard(dashboard, RulePackConfig())
+ override = next(
+ item
+ for item in resolved.panel_layout_overrides
+ if item["title_match"] == "Locks by state" and not item.get("section_match")
+ )
+ panels = [_xy_probe_panel("line")]
+
+ _apply_panel_layout_overrides_recursively(panels, [override])
+
+ esql = panels[0]["esql"]
+ assert esql["type"] == "bar"
+ assert esql["mode"] == "stacked"
+ assert esql["legend"] == {"position": "right", "visible": "show"}
+ assert not dashboard_schema_errors(panels)
+
+
+def test_layout_override_rejects_non_xy_kibana_type_override():
+ """A cross-shape request is a rule-pack error, not a silent bad emit."""
+ with pytest.raises(ValueError) as excinfo:
+ validate_rule_pack_payload(
+ {
+ "panel": {
+ "layout_overrides": [
+ {"title_match": "Uptime", "kibana_type_override": "metric"}
+ ]
+ }
+ },
+ source="probe pack",
+ )
+
+ message = str(excinfo.value)
+ assert "presentation-only" in message
+ assert "query_overrides" in message
+
+
+def test_layout_override_rejects_xy_mode_on_line_type():
+ with pytest.raises(ValueError) as excinfo:
+ validate_rule_pack_payload(
+ {
+ "panel": {
+ "layout_overrides": [
+ {
+ "title_match": "Locks by state",
+ "kibana_type_override": "line",
+ "xy_mode": "stacked",
+ }
+ ]
+ }
+ },
+ source="probe pack",
+ )
+
+ assert "no stacking mode" in str(excinfo.value)
+
+
+def test_layout_override_accepts_xy_family_type_and_stacking():
+ payload = validate_rule_pack_payload(
+ {
+ "panel": {
+ "layout_overrides": [
+ {
+ "title_match": "Locks by state",
+ "kibana_type_override": "bar",
+ "xy_mode": "stacked",
+ "legend_position": "right",
+ }
+ ]
+ }
+ }
+ )
+
+ override = payload.panel.layout_overrides[0]
+ assert override.kibana_type_override == "bar"
+ assert override.xy_mode == "stacked"
+ assert override.legend_position == "right"
+
+
+def test_skipped_layout_presentation_override_is_reported_on_the_panel():
+ """The skip is an operator-visible gap: the panel does not look the way the
+ pack asked, so it must not be reported as a clean ``migrated``."""
+ dashboard = {
+ "title": "Layout Override Probe",
+ "panels": [
+ {
+ "id": 1,
+ "type": "singlestat",
+ "title": "Total database size",
+ "targets": [{"expr": "sum(pg_database_size_bytes)", "refId": "A"}],
+ "gridPos": {"x": 0, "y": 0, "w": 4, "h": 3},
+ }
+ ],
+ }
+ rule_pack = RulePackConfig(
+ panel_layout_overrides=[
+ {"title_match": "Total database size", "xy_mode": "stacked"}
+ ]
+ )
+
+ result = translate_dashboard(
+ dashboard,
+ datasource_index="metrics-*",
+ esql_index="metrics-*",
+ rule_pack=rule_pack,
+ )
+
+ panel_result = next(
+ item
+ for item in result.panel_results
+ if item.title == "Total database size"
+ )
+ assert panel_result.status == "migrated_with_warnings", panel_result.reasons
+ assert any(
+ "presentation change was skipped" in reason for reason in panel_result.reasons
+ ), panel_result.reasons
+ assert result.migrated_with_warnings >= 1
+ assert not dashboard_schema_errors(
+ result.dashboard_ir.to_yaml_dict().get("panels") or []
+ )
+
+
def test_panel_layout_overrides_can_rename_section_title():
panels = [
{
@@ -2230,6 +3234,41 @@ def test_panel_layout_overrides_can_rename_section_title():
assert panels[0]["title"] == "Overview"
+def test_panel_layout_overrides_section_match_uses_source_title_after_rename():
+ """section_match is the Grafana row title, not the post-override Kibana title."""
+ panels = [
+ {
+ "title": "Section 1",
+ "section": {
+ "collapsed": False,
+ "panels": [
+ {
+ "title": "MySQL Uptime",
+ "position": {"x": 0, "y": 0},
+ "size": {"w": 6, "h": 6},
+ }
+ ],
+ },
+ }
+ ]
+ overrides = [
+ {"title_match": "Section 1", "title": "Overview"},
+ {
+ "title_match": "MySQL Uptime",
+ "section_match": "Section 1",
+ "position": {"x": 8, "y": 2},
+ "size": {"w": 12, "h": 8},
+ },
+ ]
+
+ _apply_panel_layout_overrides_recursively(panels, overrides)
+
+ assert panels[0]["title"] == "Overview"
+ child = panels[0]["section"]["panels"][0]
+ assert child["position"] == {"x": 8, "y": 2}
+ assert child["size"] == {"w": 12, "h": 8}
+
+
def test_curated_query_override_materializes_control_and_metric_placeholders():
class _FakeResolver:
def resolve_control_field(self, name, metric_field=None):