Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions docs/contributing/dev-commands.md
Original file line number Diff line number Diff line change
Expand Up @@ -435,6 +435,25 @@ bash scripts/run_migration.sh --skip-data
bash scripts/run_migration.sh --skip-upload
```

### Curated Pack Pin Verification

`observability_migration/adapters/source/grafana/curated_packs/registry.yaml`
pins each curated pack's exact grafana.com `gnet_revision` and canonical-JSON
`dashboard_sha256` — a maintainer provenance check (issue #350), not a
migration-time gate. Re-verify after touching a pack or its registry entry, or
before re-pinning to a newer revision (requires network; mirrors
`scripts/fetch_community_corpus.py`'s `canonical_sha256` pattern; not part of
`make test`):

```bash
.venv/bin/python scripts/verify_curated_pack_pins.py
.venv/bin/python scripts/verify_curated_pack_pins.py --gnet-id 1860
```

`tests/test_verify_curated_pack_pins.py` covers the hashing/parsing logic
offline (mocked download); `tests/test_curated_packs.py` guards the registry's
shape (required fields, unique ids/names/paths/hashes, hex-digest format).

### Schema / Lint / Layout

```bash
Expand Down
53 changes: 34 additions & 19 deletions docs/design/curated-dashboard-packs.md
Original file line number Diff line number Diff line change
Expand Up @@ -382,28 +382,43 @@ packs:
path: grafana_12776_redis
```

### At migration time — drift detection

When `resolve_pack_for_dashboard` loads a curated pack:
1. It compares the incoming dashboard JSON's revision (if present in `_grafana_meta` or dashboard JSON) against `gnet_revision`.
2. If they match → apply the pack normally, no warning.
3. If the incoming revision is newer → emit a structured warning:
```
WARNING: Curated pack grafana_12776_redis was validated against revision 6;
this dashboard is revision 9. The pack will still be applied — metric_kinds
and label_candidates remain valid across minor revisions — but new or changed
panels may not benefit from fidelity overrides. Run `obs-migrate curated-pack check`
to see what changed.
```
4. The pack is still applied — most of it (metric_kinds, label_candidates, layout for existing panels) remains correct across minor revisions. Only net-new panels fall back to the general pipeline.
### At migration time — no revision comparison (issue #350)

`resolve_pack_for_dashboard` matches a curated pack purely by `gnetId` (with a
title/tags fallback) and applies it unconditionally — it never reads or
compares the incoming dashboard JSON's revision against the registry's
`gnet_revision`, and emits no drift warning. This is intentional, not an

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This rewrite is accurate and matches the code (resolve_pack_for_dashboard does zero revision comparison).

Two leftovers later in this same file still claim the old behavior this section exists to retract:

  • File table (~line 560): rules.pydrift warning
  • Open Questions Resolved (~line 576): "Dashboard version drift: Packs pin gnet_revision; migration warns (not errors) on mismatch and still applies the pack."

That's the exact false claim this PR is correcting. Please update those two spots in the rebase so the spec is internally consistent.

oversight: an operator's real Grafana-instance export is structurally
different from a pristine `grafana.com` download at the *same* revision
(mutated `id`/`uid`/`version`/panel-`id`/etc.), so any such comparison would
mismatch on effectively every real migration and would not be a meaningful
signal either way.

`gnet_revision` and `dashboard_sha256` are instead **maintainer-verified
provenance pins** — they record which exact `grafana.com` revision a pack's
author read when writing its `pack.yaml` overrides, re-checkable offline
against a fresh `grafana.com` download with
`python scripts/verify_curated_pack_pins.py` (network required; not part of
`make test` — see `docs/contributing/dev-commands.md`). The actual risk this
guards against — a pack silently missing dashboard content because the
upstream dashboard changed since the pack was authored — is caught at
migration time by a different, per-panel mechanism: the translator compares
each panel's source PromQL metrics against what survives into the final
emitted query and downgrades status/confidence with a warning when one is
dropped (`docs/sources/grafana.md`), independent of whether the revision pin
is current.

### Pack update workflow

When a significant new revision of a community dashboard is published:
1. `obs-migrate curated-pack diff --gnet-id 12776` — shows which panels changed vs. the pinned revision.
1. Download the new revision and diff it by hand against the pinned one (no
dedicated CLI command exists for this — `curl` the
`grafana.com/api/dashboards/{id}/revisions/{revision}/download` endpoint
for both revisions and compare).
2. Author updates the `fidelity_manifest.yaml` for changed/added panels.
3. Update `pack.yaml` / `plugin.py` if new metrics or panel types appear.
4. Update `gnet_revision` and `dashboard_sha256` in `registry.yaml`.
4. Update `gnet_revision` and `dashboard_sha256` in `registry.yaml`, then
confirm with `python scripts/verify_curated_pack_pins.py --gnet-id <id>`.
5. Re-run the full gate stack.

### What never breaks across revisions
Expand Down Expand Up @@ -542,10 +557,10 @@ For the Redis 12776 pack specifically:
| `parity-rig/curated/grafana_12776_redis/docker-compose.yml` | Docker test stack (ES + Kibana + Grafana + Redis + redis_exporter + Prometheus) |
| `parity-rig/curated/grafana_12776_redis/grafana_provisioning/` | Grafana dashboard + datasource provisioning |
| `parity-rig/curated/grafana_12776_redis/prometheus.yml` | Prometheus scrape + remote_write config |
| `observability_migration/adapters/source/grafana/rules.py` | `resolve_pack_for_dashboard()`, `_load_curated_pack_for()`, drift warning |
| `observability_migration/adapters/source/grafana/rules.py` | `resolve_pack_for_dashboard()`, `_load_curated_pack_for()` |
| `observability_migration/adapters/source/grafana/cli.py` | Call `resolve_pack_for_dashboard` per dashboard; `--no-curated-packs` flag |
| `observability_migration/app/cli.py` | Forward `--no-curated-packs` in unified CLI |
| `tests/test_curated_packs.py` | Registry lookup, merge correctness, drift detection, no-curated-packs opt-out |
| `tests/test_curated_packs.py` | Registry lookup, merge correctness, provenance-pin shape, manifest-vs-registry consistency, no-curated-packs opt-out |
| `tests/curated/test_grafana_12776_redis.py` | Redis pack fixture tests (offline) |

---
Expand All @@ -558,7 +573,7 @@ For the Redis 12776 pack specifically:
- **User override**: User `--rules-file` always wins over curated pack on key collision.
- **No panel abandoned**: All panels get best-effort treatment. BEST_EFFORT replaces NOT_FEASIBLE — find the closest Kibana alternative, document the delta.
- **Layout curation**: Curated packs control Kibana grid layout, not just queries/visuals.
- **Dashboard version drift**: Packs pin `gnet_revision`; migration warns (not errors) on mismatch and still applies the pack.
- **Dashboard version drift**: Packs pin `gnet_revision`/`dashboard_sha256` as maintainer-verified provenance only; migration never compares them against the incoming dashboard and emits no drift warning (issue #350 — see "At migration time — no revision comparison" above). A pack that has fallen behind its upstream dashboard is caught per-panel instead, by the dropped-source-metric disclosure.

---

Expand Down
11 changes: 11 additions & 0 deletions docs/sources/grafana.md
Original file line number Diff line number Diff line change
Expand Up @@ -124,6 +124,17 @@ without it they no-op rather than guess. Metrics listed in the pack's
hand-written override so the rest of the panel can still render; those
omissions are not reported as pack gaps.

Each pack is registered in `curated_packs/registry.yaml` with a
`gnet_revision` and `dashboard_sha256` — maintainer-verified provenance pins
recording the exact grafana.com revision the pack authors read, re-checkable
with `scripts/verify_curated_pack_pins.py`
(`docs/contributing/dev-commands.md`). These are not enforced against operator
dashboards at migration time: a pristine grafana.com download differs
structurally from any real Grafana-instance import/export, so that comparison
would mismatch on every real migration. A pack silently missing dashboard
content is instead caught per-panel by the dropped-source-metric disclosure
described above.

The console pipeline is **5 stages**, not 7: `[1/5] Extracting dashboards`,
`[2/5] Translating dashboards`, `[3/5] Verification-packet ES|QL validation`,
`[4/5] Writing native Dashboard-as-Code review artifacts`, `[5/5] Generating
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,10 @@

schema_version: 1
gnet_id: 11835
gnet_revision: 4
# gnet_revision 4 (previously pinned here and in registry.yaml) does not
# exist on grafana.com; re-pinned to revision 1, the only real revision
# (issue #350). See registry.yaml's entry for this pack for the full caveat.
gnet_revision: 1
dashboard_title: "Redis Dashboard for Prometheus Redis Exporter (helm stable/redis-ha)"

panels:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,24 @@
# 1. Exact gnetId match (preferred — reliable even when title differs)
# 2. title_hint + tags_hint fallback (for dashboards where gnetId was stripped on import)
#
# `gnet_revision` and `dashboard_sha256` are maintainer-verified provenance
# pins, not a migration-time enforcement mechanism: they record which exact
# grafana.com revision the pack authors read when writing `pack.yaml`, so a
# maintainer can re-verify a pack still matches its stated source (or
# deliberately re-pin it) as the upstream dashboard evolves. `dashboard_sha256`
# is the SHA-256 of the *canonical* (sort_keys, compact-separator) JSON
# re-serialization of that revision's `grafana.com` download — see
# `canonical_sha256` in scripts/verify_curated_pack_pins.py, the same pattern
# `scripts/fetch_community_corpus.py` uses for the benchmark corpus pins.
# Verify with `python scripts/verify_curated_pack_pins.py` (network required;
# not part of `make test` — see docs/contributing/dev-commands.md). These pins
# are NOT checked against operator dashboards at migration time: a pristine
# grafana.com download differs structurally from any Grafana-instance
# import/export (mutated id/uid/version/... fields), so that comparison would
# mismatch on every real migration. A pack silently missing dashboard content
# is instead caught per-panel by the dropped-source-metric detection in
# panels.py (see docs/sources/grafana.md).
#
# To add a new pack: follow the Curation Playbook in docs/design/curated-dashboard-packs-plan.md.

packs:
Expand All @@ -18,7 +36,7 @@ packs:
tags_hint: ["redis", "prometheus"]
path: grafana_763_redis_exporter
gnet_revision: 6
dashboard_sha256: "dcee8585ef010e7569dab8c776e48e3a6ee5ebe9c03a1510891488b714faeb44"
dashboard_sha256: "8c34e52a08effbfa559b3c395fafea94750f072398d622a5329b3db6901b07b1"
description: "Redis open-source (oliver006/redis_exporter) — counter/gauge classification, label map, layout"

- gnet_id: 1860
Expand All @@ -27,7 +45,7 @@ packs:
tags_hint: ["prometheus"]
path: grafana_1860_node_exporter_full
gnet_revision: 37
dashboard_sha256: "d4313569a6a74471de08e3160c674e4b5c015f1385f385170232734f9124ea63"
dashboard_sha256: "2d8c3321510e49bb0dec9cc0d0f7270136ad0b29579e14d27e1e8819464c7670"
description: "Node Exporter Full — suffix-less counter typing plus a curated process-memory repair for Kibana"

- gnet_id: 18405
Expand All @@ -36,7 +54,7 @@ packs:
tags_hint: []
path: grafana_18405_redis_enterprise
gnet_revision: 1
dashboard_sha256: "b5c96cbc62efe9d76ea43d9214a896aacab8e5de4b31d71b957328c262b1946d"
dashboard_sha256: "f341b5b1f3596eec5a902c6e921528a4a2c9300ec21c3b8bc045edee2fbbda46"
description: "Redis Enterprise cluster monitoring (bdb_* Prometheus metrics) — stat panels, cluster/db variables"

- gnet_id: 18406
Expand All @@ -45,7 +63,7 @@ packs:
tags_hint: ["RLEC"]
path: grafana_18406_redis_cloud
gnet_revision: 2
dashboard_sha256: "b5f9f8b76ade994accd6cecd9a9c32f22e5e8ae46aa889a53bc9723db28a0ec9"
dashboard_sha256: "88c161611f6805cf0b4516864cec10015f523704f69ac781ca04f2e0da09e13c"
description: "Redis Cloud subscription monitoring (bdb_* Prometheus metrics) — stat panels, cluster/bdb variables"

- gnet_id: 14091
Expand All @@ -54,14 +72,20 @@ packs:
tags_hint: []
path: grafana_14091_redis_exporter_quickstart
gnet_revision: 1
dashboard_sha256: "ad46c7c61701f7c5ce55bae9daa56fecde0afa30980d2603b5919370e11a5f85"
dashboard_sha256: "4e92754d0aaec2466bbcd37cf92b7ca551eb5e52a9368b55edd207acf4e1e86d"
description: "Grafana Labs Redis Exporter Quickstart — hit-ratio plugin, renamed fragmentation metric, label map"

- gnet_id: 11835
name: grafana_11835_redis_exporter_helm
title_hint: "Redis Dashboard for Prometheus Redis Exporter (helm stable/redis-ha)"
tags_hint: ["redis", "prometheus"]
path: grafana_11835_redis_exporter_helm
gnet_revision: 4
dashboard_sha256: "bd67264fd0d73f633a67b3b8f9d4e5f2e527c569e0e946e8f511168922b18b96"
# NOTE (issue #350): the previously pinned gnet_revision 4 does not exist
# on grafana.com — the dashboard has only ever had revision 1 (verified
# via GET /api/dashboards/11835/revisions). Re-pinned to the only real
# revision. This cannot be independently confirmed to be the exact
# content the pack authors originally read; re-review pack.yaml's
# overrides against this revision if any mismatch surfaces.
gnet_revision: 1
dashboard_sha256: "9ad90c516c051b5592296a587f217994a9e8d6aee3e4c78ec990075c1d3e3d45"
description: "Redis helm stable/redis-ha (oliver006/redis_exporter) — counter/gauge classification, label map"
146 changes: 146 additions & 0 deletions scripts/verify_curated_pack_pins.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,146 @@
#!/usr/bin/env python3
# Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one or more contributor license agreements.
# SPDX-License-Identifier: Elastic-2.0

"""Verify the curated-pack provenance pins in registry.yaml (issue #350).

Each entry in
``observability_migration/adapters/source/grafana/curated_packs/registry.yaml``
pins the exact grafana.com ``gnet_revision`` and canonical-JSON ``sha256`` the
pack authors read when writing that pack's ``pack.yaml`` overrides. This is a
**maintainer provenance check**, not a migration-time gate: it lets a
maintainer confirm a pack still matches its stated source dashboard (or
deliberately re-pin it) as the upstream dashboard evolves on grafana.com. It
is intentionally NOT run against operator dashboards at migration time -- a
pristine grafana.com download differs structurally from any real Grafana
instance's import/export (mutated ``id``/``uid``/``version``/etc.), so that
comparison would mismatch on every real migration. The actual risk a pin
guards against (a pack silently missing dashboard content) is instead caught
per-panel by the dropped-source-metric detection in ``panels.py``.

Mirrors the download + ``canonical_sha256`` pattern already proven by
``scripts/fetch_community_corpus.py`` for the benchmark corpus pins.

Requires network access. Not part of ``make test`` -- run manually (or from a
maintainer CI job) after touching a curated pack or its registry entry::

python scripts/verify_curated_pack_pins.py
python scripts/verify_curated_pack_pins.py --gnet-id 1860
"""
from __future__ import annotations

import argparse
import hashlib
import json
import sys
import urllib.request
from pathlib import Path
from typing import Any

import yaml

REPO_ROOT = Path(__file__).resolve().parents[1]
DEFAULT_REGISTRY = (
REPO_ROOT
/ "observability_migration"
/ "adapters"
/ "source"
/ "grafana"
/ "curated_packs"
/ "registry.yaml"
)
DOWNLOAD_URL = "https://grafana.com/api/dashboards/{id}/revisions/{revision}/download"


def _download(url: str, timeout: int = 30) -> bytes:
req = urllib.request.Request(url, headers={"User-Agent": "obs-migrate-curated-pack-pin-verify"})
with urllib.request.urlopen(req, timeout=timeout) as response:
return response.read()


def canonical_sha256(raw: bytes) -> str:
"""Checksum a canonical (sort_keys, compact) form of the dashboard JSON.

grafana.com re-serializes the same revision with differing key order /
whitespace across requests, so raw-byte hashes are not stable. Canonical
JSON is, which makes the pin tamper-evident without being
serialization-fragile. Identical to
``scripts/fetch_community_corpus.py::canonical_sha256`` by design -- both
pin the same kind of grafana.com download.
"""
return hashlib.sha256(
json.dumps(json.loads(raw), sort_keys=True, separators=(",", ":")).encode()
).hexdigest()


def load_packs(registry_path: Path) -> list[dict[str, Any]]:
data = yaml.safe_load(registry_path.read_text()) or {}
packs = data.get("packs") or []
if not isinstance(packs, list):
raise ValueError(f"{registry_path}: 'packs' must be a list, got {type(packs).__name__}")
for entry in packs:
if not isinstance(entry, dict):
raise ValueError(f"{registry_path}: each pack entry must be a mapping, got {entry!r}")
return list(packs)


def verify_pack(pack: dict[str, Any]) -> tuple[bool, str]:
gnet_id = pack.get("gnet_id")
revision = pack.get("gnet_revision")
expected = str(pack.get("dashboard_sha256") or "")
name = pack.get("name") or f"gnet_{gnet_id}"
if not gnet_id or not revision:
return False, f"{name}: missing gnet_id/gnet_revision in registry entry"
url = DOWNLOAD_URL.format(id=gnet_id, revision=revision)
try:
raw = _download(url)
digest = canonical_sha256(raw)
except Exception as exc:
# Covers both transport failures (network/HTTP) and a malformed/
# non-JSON response body -- either way this pack's pin cannot be
# confirmed, and it must not abort the rest of the registry's checks.
return False, f"{name} (id={gnet_id}, rev={revision}): verification error: {exc}"
if digest != expected:
return False, (
f"{name} (id={gnet_id}, rev={revision}): sha256 mismatch "
f"(registry has {expected[:12] or '(empty)'}, grafana.com now hashes to {digest[:12]})"
)
return True, f"{name} (id={gnet_id}, rev={revision}): OK"


def verify_registry(registry_path: Path, *, gnet_id: int | None = None) -> int:
packs = load_packs(registry_path)
if gnet_id is not None:
packs = [p for p in packs if p.get("gnet_id") == gnet_id]
if not packs:
print(f"No registry entry with gnet_id={gnet_id}", file=sys.stderr)
return 1
elif not packs:
# An empty registry is never a meaningful "all pins verified" --
# most likely a parsing/path problem that would otherwise silently
# print "0/0 pins verified" and exit 0.
print(f"No pack entries found in {registry_path}", file=sys.stderr)
return 1
failures = 0
for pack in packs:
ok, message = verify_pack(pack)
print(("OK " if ok else "FAIL ") + message)
if not ok:
failures += 1
print(f"--- {len(packs) - failures}/{len(packs)} pins verified ---")
return 1 if failures else 0


def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--registry", type=Path, default=DEFAULT_REGISTRY)
parser.add_argument(
"--gnet-id", type=int, default=None,
help="Verify only the entry with this gnet_id (default: verify every entry).",
)
args = parser.parse_args(argv)
return verify_registry(args.registry, gnet_id=args.gnet_id)


if __name__ == "__main__":
raise SystemExit(main())
Loading
Loading