Grafana `interval` template variables are dropped with no control and no warning

[giorgi-imerlishvili-elastic]

Summary

Grafana template variables of type interval are dropped during migration. No Kibana control is emitted for them, nothing in the migrated artifact references them, and no warning is raised — even though the variable may be controlling the rate window of every query on the dashboard.

Grafana community dashboard 9852 "node-exporter disk graphs" declares:

{ "name": "RateInterval", "type": "interval", "query": "20s,1m,5m" }

and 16 of its targets use it as the rate window, e.g.

rate(node_disk_written_bytes_total{instance=~"$Node", device!="sr0", device=~"$Disk"}[$RateInterval])

After migration:

• Controls emitted: Node, CPU, Disk. No RateInterval.
• The string RateInterval does not appear anywhere in the native dashboard artifact (it survives only in migration_report.json / verification_packets.json as a copy of the original PromQL).
• CONTROL WARNINGS (3) covers the multi-select semantics of Node, CPU and Disk. There is no warning that a fourth variable was discarded.

The operator loses a control that was on the source dashboard, and is not told.

Why this is more than a missing dropdown

In Grafana the rate window and the display resolution are independent: [$RateInterval] defaults to 1m, so with a 15s scrape each point averages ~4 samples and the line is smooth, regardless of how wide the chart is.

In the migrated panel the rate window is implicitly whatever TBUCKET chose. On a 15-minute range TBUCKET(100, ?_tstart, ?_tend) resolves to 10-second buckets (confirmed in the response metadata: "_meta":{"bucket":{"interval":10,"unit":"second"}}), which is narrower than the 15s scrape interval. The same panel that is smooth in Grafana comes out stair-stepped with periodic gaps in Kibana, and the operator has no 1m/5m option to smooth it back.

So dropping the variable does not just remove a widget — it silently transfers control of the rate window to the bucket heuristic, and the two are not equivalent.

Environment

• Branch feat/curated-dashboard-packs (PR feat!: curated dashboard packs, PromQL fidelity, and native-only dashboard artifacts #346) at 5db742f
• Kibana / Elasticsearch 9.5.0-SNAPSHOT
• Source: Grafana community dashboard 9852, Prometheus + node_exporter, 15s scrape
• --field-profile prometheus_native, no curated pack applies

Reproduction

obs-migrate migrate \
  --source grafana \
  --input-mode api \
  --grafana-url "$GRAFANA_URL" \
  --field-profile prometheus_native \
  --output-dir ./out

Then:

grep -c RateInterval ./out/dashboards/native/*.json     # 0
python -c "import json,glob; \
  print([c['config']['title'] for c in json.load(open(glob.glob('./out/dashboards/native/*.json')[0]))['payload']['pinned_panels']])"
# ['Node', 'CPU', 'Disk']

Suggested fix

Two parts, and the first matters more than the second:

1. Disclose it. An interval variable that is referenced by panel queries and then dropped should produce a warning in the same list as the other control warnings, naming the variable and what now determines the rate window. Silent loss of a source control is the part that conflicts with the "degrade gracefully, do not hide semantic gaps" rule in AGENTS.md.
2. Optionally support it. The values are a fixed list (20s,1m,5m), so it could map to a Kibana ES|QL control bound into the bucket/window expression, giving back the smoothing choice.

Related

• ES|QL time buckets should match Grafana auto vs fixed interval #316 — ES|QL time buckets should match Grafana auto vs fixed interval. That issue is about panel resolution; this one is about a distinct user-facing variable that controls the rate window. They interact: with the variable gone, bucket width is the only thing left setting the window.
• TBUCKET(100, ?_tstart, ?_tend) yields sub-scrape-interval buckets, so IRATE/RATE panels are empty on normal time ranges #348 — TBUCKET(100, ?_tstart, ?_tend) producing sub-scrape-interval buckets, which is what the dropped variable would otherwise have masked
• Found while manually testing PR feat!: curated dashboard packs, PromQL fidelity, and native-only dashboard artifacts #346 with dashboard 9852

[su-jin1425]

Hi I’d like to work on this issue. I’ll review it and share my plan ASAP.

[su-jin1425]

I’m planning to first trace how Grafana template variables are parsed, referenced by panel queries, and converted into migration controls. I’ll identify interval variables that are referenced by queries but are currently dropped during migration, then add a control warning that names the variable and explains that its rate-window behavior is no longer preserved. I’ll add regression tests covering the interval variable, its query reference, the generated warning, and the existing behavior for other variable types. After that, I’ll run the migration against the provided dashboard reproduction to verify that the variable is no longer silently lost.

If any maintainer or code reviewer is reading this, please /assign this issue to me or let me know. I’d be happy to take it up and start working on it. Thanks!