feat(verifier): dashboard fidelity verification framework

AGENTS.md

@@ -45,6 +45,28 @@ and the license/SBOM refresh. Prefer `make` so the environment matches CI.
 - Do not duplicate long command walkthroughs outside `docs/command-contract.md`.
 - Do not commit secrets or generated local artifacts.
 - Preserve the existing "degrade gracefully" behavior for unsupported translations instead of hiding semantic gaps.
+- For dashboard migration fixes, prove the generated YAML matches `docs/dashboards/schema.json`
+  and the compiled/uploaded Kibana saved object. Do not infer schema support
+  from Kibana UI affordances alone. In particular, Lens XY YAML (`line`, `area`,
+  `bar`) has a single `breakdown`; use a synthetic composite field when multiple
+  source labels must define one series identity. Multi-breakdown arrays are for
+  datatable/pie/treemap-style schemas unless the schema/compiler prove otherwise.
+- For user-facing dashboard correctness claims, validate against real migrated
+  artifacts and uploaded dashboards: inspect YAML, compiled NDJSON/saved objects,
+  run scoped smoke/direct `_query` checks, and browser-check a clean view-mode
+  Kibana session. Clear stale dashboard edit state before trusting browser output.
+- For dashboard-regression fixes, run the layered verifier gates documented in
+  `docs/command-contract.md`: `verifier.live_validate` (runtime ES|QL),
+  `verifier.dashboards_api` (typed Kibana dashboard contract), `obs-migrate
+  compare` + `verifier.corpus_gate` (semantic parity), and
+  `verifier.benchmark_gate` (PM benchmark-history regression guard). Track both
+  percentages and denominators (`dashboards`, `panels_total`,
+  `verification_total`), including datasource-filtered Grafana slices when the
+  benchmark UI is filtered.
+- Grow benchmark coverage through pinned, stratified manifests generated by
+  `verifier.corpus_manifest` (top dashboards, long-tail slices, datasource
+  quotas, and explicit bug seeds). Avoid unpinned "top N today" corpora as merge
+  gates because marketplace changes can look like code regressions.
 - Skills are mirrored in `.claude/skills/` and `.cursor/skills/` (one `SKILL.md` per skill in each tree). When you add or edit a skill, update **both** copies. They are byte-identical **except** self-referential path prefixes — `~/.claude/...` in the `.claude` copy vs `~/.cursor/...` in the `.cursor` copy — so don't blindly `cp` a skill that links to other skills; rewrite those prefixes for the destination tree.
 
 ## Commit And Push Workflow (For Agents)

CLAUDE.md

@@ -21,6 +21,27 @@ this repo as the single source of truth for it. (See the Naming note in
 - Build / test / lint: see `AGENTS.md` (use `make test`, `make lint`, `make typecheck`).
 - Preserve "degrade gracefully" behavior for unsupported translations — do not silently hide semantic gaps.
 - Do not commit secrets or generated local artifacts.
+- Dashboard migration fixes must be checked against the schema, compiled saved
+  object, and uploaded Kibana behavior. Do not infer YAML support from Kibana UI
+  controls alone: Lens XY YAML supports one `breakdown`, while multi-breakdown
+  arrays are for schemas such as datatable/pie/treemap unless
+  `docs/dashboards/schema.json` and the compiler prove otherwise.
+- Before claiming migrated dashboards render correctly, validate real artifacts:
+  generated YAML, compiled NDJSON/saved object, scoped smoke or direct `_query`,
+  and a clean view-mode browser session. Clear stale dashboard edit state before
+  trusting browser observations.
+- For dashboard-regression work, use the layered verifier gates documented in
+  `docs/command-contract.md`: `verifier.live_validate` for runtime ES|QL errors,
+  `verifier.dashboards_api` for typed Kibana UI-contract validation,
+  `obs-migrate compare` plus `verifier.corpus_gate` for semantic parity, and
+  `verifier.benchmark_gate` for PM benchmark-history regressions. Do not rely
+  on a single migrated/clean percentage; also watch denominator drops
+  (`panels_total`, `dashboards`, `verification_total`) and filtered datasource
+  slices.
+- When growing benchmark coverage, prefer pinned stratified manifests from
+  `verifier.corpus_manifest` (top dashboards + long-tail + datasource quotas +
+  bug seeds) over an unpinned "top N today" sample. Use small deterministic PR
+  gates and larger/nightly corpus gates.
 - Skills live in both `.claude/skills/` and `.cursor/skills/` — edit both copies in lockstep (see the mirroring rule in `AGENTS.md` for the `.claude`↔`.cursor` path-prefix caveat).
 
 ## Commit Workflow

docs/command-contract.md

@@ -636,6 +636,90 @@ then clean up:
   --confirm
 ```
 
+### Verification And Benchmark Gates
+
+The `parity-rig/verifier/` tools are repo-oriented correctness gates used by
+development and CI. They are intentionally layered: each gate answers a different
+question, and no single gate is sufficient for "the dashboard is correct".
+
+| Tool | Input | What it proves | Typical gate |
+|---|---|---|---|
+| `verifier.live_validate` | `migration_report.json` | Elasticsearch accepts the emitted ES|QL (`real_bug` vs `data_gap`) | no `real_bug` |
+| `verifier.dashboards_api` | `migration_report.json` + Kibana | Kibana's typed Dashboards API accepts the mapped panel payload | no `dashboards_api_rejected` |
+| `obs-migrate compare` | `verification_packets.json` + seeded data | Native PromQL and translated ES|QL are numerically close | no `FAIL`/`ERROR`; bounded `SHAPE_PASS` |
+| `verifier.corpus_gate` | `obs-migrate compare` report(s) | Frozen semantic corpus does not regress | configured budgets |
+| `verifier.benchmark_gate` | PM `benchmark_history.json` | Migration success metrics do not drop vs compatible baseline | configured budgets |
+| `verifier.mutations` | `migration_report.json` | The invariant verifier catches deliberate corruptions | all mutations pass |
+| `verifier.lens_fixtures` | LensConfigBuilder fixture JSON | Authoritative Lens-as-code fixtures exist for required chart families | coverage complete |
+| `verifier.corpus_manifest` | Grafana catalog + datasource map | Larger benchmark corpus is pinned/stratified/reproducible | committed manifest |
+
+Examples:
+
+```bash
+# Runtime ES|QL oracle: catches invalid emitted ES|QL that compile/lint miss.
+PYTHONPATH=parity-rig .venv/bin/python -m verifier.live_validate \
+  --migration-out migration_output/dashboards \
+  --es-url "$ELASTICSEARCH_ENDPOINT" \
+  --api-key "$KEY" \
+  --fail-on-bug
+
+# Typed Kibana UI contract: validates mapped panels against /api/dashboards.
+PYTHONPATH=parity-rig .venv/bin/python -m verifier.dashboards_api \
+  --migration-out migration_output/dashboards \
+  --kibana-url "$KIBANA_ENDPOINT" \
+  --api-key "$KEY" \
+  --fail-on-error
+
+# Semantic corpus gate over compare reports.
+PYTHONPATH=parity-rig .venv/bin/python -m verifier.corpus_gate \
+  --report comparison_report.json \
+  --max-fail 0 \
+  --max-error 0 \
+  --max-shape-pass 25
+
+# PM benchmark-history gate. Compare the latest run to the most recent
+# compatible different CLI hash (same G/D config and schema-discovery class).
+PYTHONPATH=parity-rig .venv/bin/python -m verifier.benchmark_gate \
+  --history benchmark_history.json \
+  --max-drop-pp 0.5 \
+  --max-count-drop 5 \
+  --max-duration-increase-pct 100
+
+# Same gate, but scoped like the PM UI's datasource filters.
+PYTHONPATH=parity-rig .venv/bin/python -m verifier.benchmark_gate \
+  --history benchmark_history.json \
+  --source grafana \
+  --grafana-datasource prometheus \
+  --grafana-datasource-map grafana-datasources.json \
+  --max-drop-pp 0.5 \
+  --max-count-drop 5
+
+# Build a bigger pinned corpus manifest without introducing marketplace noise.
+PYTHONPATH=parity-rig .venv/bin/python -m verifier.corpus_manifest \
+  --grafana-catalog dashboards.json \
+  --grafana-datasource-map grafana-datasources.json \
+  --top 500 \
+  --long-tail tail_500_2000:500:2000:100 \
+  --datasource-quota prometheus=100 \
+  --datasource-quota loki=50 \
+  --bug-seed 1860 \
+  --output corpus.manifest.json
+```
+
+Regression-gate guidance:
+
+- Use `benchmark_gate` for the PM trend numbers (`dashboard_migration_pct`,
+  `dashboard_clean_pct`, `panel_migration_pct`, `panel_clean_pct`,
+  `panel_verified_pct`, and optional duration). It also checks denominator drops
+  (`dashboards`, `panels_total`, `verification_total`) so stable percentages
+  cannot hide a smaller corpus or reduced verification coverage.
+- Keep PR gates smaller and deterministic. Use a pinned manifest from
+  `corpus_manifest` plus bug seeds. Run the larger stratified corpus nightly or
+  before risky translator changes.
+- A `benchmark_gate` "no compatible baseline" result is not a pass on quality;
+  it means the run changed config/schema class enough that the gate cannot make
+  a fair comparison. Establish a new baseline before relying on trend decisions.
+
 ### Remove Sample Data
 
 `obs-migrate remove-sample-data` tears down what `seed-sample-data` created. It

parity-rig/verifier/__init__.py

@@ -30,10 +30,24 @@
 
 from __future__ import annotations
 
+from .invariants import (
+    Finding,
+    InvariantCategory,
+    Severity,
+    lint_report,
+)
 from .records import (
     DRIFT_AXES,
     PanelRecord,
     Verdict,
 )
 
-__all__ = ["DRIFT_AXES", "PanelRecord", "Verdict"]
+__all__ = [
+    "DRIFT_AXES",
+    "Finding",
+    "InvariantCategory",
+    "PanelRecord",
+    "Severity",
+    "Verdict",
+    "lint_report",
+]