Summary
When a panel has several targets with different legendFormat templates, the translator builds a single legend column from the first target's template and applies it to every series. Series belonging to targets 2..N are then labelled with another metric's name.
In Grafana community dashboard 9852 "node-exporter disk graphs", the Disk IO panel has three targets:
| # |
legendFormat |
metric |
| 0 |
Weighted IO time {{ device }} {{ instance }} |
node_disk_io_time_weighted_seconds_total |
| 1 |
Write time {{ device }} {{ instance }} |
node_disk_write_time_seconds_total |
| 2 |
Read time {{ device }} {{ instance }} |
node_disk_read_time_seconds_total |
The emitted ES|QL keeps all three metrics (good — this is the multi-target fusion from #311 working), but only one legend expression, hard-coded to target 0's literal text:
| STATS Weighted_IO_time = AVG(RATE(metrics.node_disk_io_time_weighted_seconds_total)),
Write_time = AVG(RATE(metrics.node_disk_write_time_seconds_total)),
Read_time = AVG(RATE(metrics.node_disk_read_time_seconds_total))
BY time_bucket = TBUCKET(100, ?_tstart, ?_tend), labels.device, labels.instance
| EVAL legend = CONCAT("Weighted IO time ", COALESCE(TO_STRING(`labels.device`), ""), " ", COALESCE(TO_STRING(`labels.instance`), ""))
Lens uses legend as the breakdown and appends the column name, so the rendered legend reads:
Weighted IO time vda node-1:9100 - Weighted IO time
Weighted IO time vda node-1:9100 - Write time <-- mislabelled
Weighted IO time vda node-1:9100 - Read time <-- mislabelled
Grafana shows:
Weighted IO time vda node-1:9100
Write time vda node-1:9100
Read time vda node-1:9100
Every series on the panel is prefixed "Weighted IO time". An operator reading the legend — or a screenshot of it — will attribute six of the nine series to the wrong metric. The suffix does carry the right column name, so the information is technically present, but the leading text actively contradicts it.
Why some panels are unaffected
The Memory panel on the same dashboard is correct (node-1:9100 - Memory active, node-1:9100 - Memory buffers, …). Its four targets differ only by a static prefix over the same {{ instance }} label, so the translator emits no EVAL legend at all and lets the column names carry the distinction. The bug appears when the per-target templates differ and contain label placeholders, forcing the CONCAT path.
Environment
Reproduction
obs-migrate migrate \
--source grafana \
--input-mode api \
--grafana-url "$GRAFANA_URL" \
--field-profile prometheus_native \
--es-url "$ES_URL" \
--es-api-key "$ES_API_KEY" \
--kibana-url "$KIBANA_URL" \
--kibana-api-key "$KIBANA_API_KEY" \
--validate --upload --ensure-data-views
Open the uploaded dashboard and read the Disk IO legend.
The panel is reported as migrated with confidence 0.85 and gate Green, so nothing signals the mislabelling.
Suggested fix
Resolve the legend per target rather than once per panel. Concretely, either:
- emit one legend expression per value column and let Lens pair column → label, or
- when the templates differ, drop the static prefix from
EVAL legend and keep only the label-derived part (vda node-1:9100), so the column name supplies the metric identity and the result reads vda node-1:9100 - Write time.
The second is a small change and already matches what the Memory panel does correctly today.
Related
Summary
When a panel has several targets with different
legendFormattemplates, the translator builds a singlelegendcolumn from the first target's template and applies it to every series. Series belonging to targets 2..N are then labelled with another metric's name.In Grafana community dashboard 9852 "node-exporter disk graphs", the
Disk IOpanel has three targets:legendFormatWeighted IO time {{ device }} {{ instance }}node_disk_io_time_weighted_seconds_totalWrite time {{ device }} {{ instance }}node_disk_write_time_seconds_totalRead time {{ device }} {{ instance }}node_disk_read_time_seconds_totalThe emitted ES|QL keeps all three metrics (good — this is the multi-target fusion from #311 working), but only one legend expression, hard-coded to target 0's literal text:
Lens uses
legendas the breakdown and appends the column name, so the rendered legend reads:Grafana shows:
Every series on the panel is prefixed "Weighted IO time". An operator reading the legend — or a screenshot of it — will attribute six of the nine series to the wrong metric. The suffix does carry the right column name, so the information is technically present, but the leading text actively contradicts it.
Why some panels are unaffected
The
Memorypanel on the same dashboard is correct (node-1:9100 - Memory active,node-1:9100 - Memory buffers, …). Its four targets differ only by a static prefix over the same{{ instance }}label, so the translator emits noEVAL legendat all and lets the column names carry the distinction. The bug appears when the per-target templates differ and contain label placeholders, forcing theCONCATpath.Environment
feat/curated-dashboard-packs(PR feat!: curated dashboard packs, PromQL fidelity, and native-only dashboard artifacts #346) at5db742f9.5.0-SNAPSHOTnode_exporter--field-profile prometheus_native, no curated pack appliesReproduction
Open the uploaded dashboard and read the
Disk IOlegend.The panel is reported as
migratedwith confidence 0.85 and gate Green, so nothing signals the mislabelling.Suggested fix
Resolve the legend per target rather than once per panel. Concretely, either:
EVAL legendand keep only the label-derived part (vda node-1:9100), so the column name supplies the metric identity and the result readsvda node-1:9100 - Write time.The second is a small change and already matches what the
Memorypanel does correctly today.Related