Summary
The template-variable guardrail the translator emits for multi-select controls fails to compile whenever the bound control values are numeric. The panel renders a red error box instead of a chart.
Grafana community dashboard 9852 "node-exporter disk graphs" has a CPU template variable populated from label_values(node_cpu_seconds_total, cpu), so its values are 0 and 1. The migrated IO Wait per core panel fails with:
Unexpected error from Elasticsearch: verification_exception - Found 2 problems
line 2:33: second argument of [MV_CONTAINS(?CPU, ".*")] must be [integer], found value [".*"] type [keyword]
line 2:60: second argument of [MV_CONTAINS(?CPU, labels.cpu)] must be [integer], found value [labels.cpu] type [keyword]
The emitted guard clause is:
| WHERE (MV_COUNT(?CPU) == 0 OR MV_CONTAINS(?CPU, ".*") OR MV_CONTAINS(?CPU, labels.cpu))
When the selected option values all look numeric, Kibana binds ?CPU as an integer multi-value parameter. Both MV_CONTAINS arms then fail type-checking — one against the string literal ".*", one against the keyword field labels.cpu — and because this is a verification (compile-time) error, the whole query fails. The MV_COUNT(?CPU) == 0 short-circuit does not help; the statement never compiles.
This is not specific to CPU indices. Any Grafana variable whose label values are numeric hits it: CPU/core IDs, ports, PIDs, HTTP status codes, queue or shard numbers, disk numbers.
Environment
Reproduction
obs-migrate migrate \
--source grafana \
--input-mode api \
--grafana-url "$GRAFANA_URL" \
--field-profile prometheus_native \
--es-url "$ES_URL" \
--es-api-key "$ES_API_KEY" \
--kibana-url "$KIBANA_URL" \
--kibana-api-key "$KIBANA_API_KEY" \
--validate --upload --ensure-data-views
Then, in the uploaded dashboard:
- Open it. The
CPU control defaults to all options selected, including the synthetic .* entry. The panel renders, because the presence of .* keeps the parameter a keyword list.
- Deselect
.*, leaving only 0, 1 — a normal operator action, since .* is the tool's stand-in for Grafana's "All".
- Reload the dashboard.
IO Wait per core now shows the verification_exception above instead of a chart.
Why the existing gates miss it
--validate reported 13 queries: 13 passed, 0 failed for this dashboard, and the pre-upload verification gate was 8 Green / 5 Yellow / 0 Red. Validation binds the parameter differently from a live control, so the type conflict never appears offline. Only a browser session with a real control selection reproduces it — the interaction-audit case described in AGENTS.md.
Suggested fix
Make the guardrail type-stable regardless of how Kibana infers the control parameter type. Options, roughly in order of preference:
- Coerce both sides explicitly, e.g. compare
TO_STRING of the parameter elements against the keyword field, so a numeric binding cannot change the comparison type.
- Force the control's option list to keyword by casting in the control query (it already does
MV_APPEND(".*", labels.cpu), which is keyword-typed at generation time but does not survive Kibana's re-inference of the selected values).
- Failing both, detect numeric-valued label variables at translation time and emit a different filter shape rather than one that cannot compile.
Whatever the fix, the .* sentinel and the field comparison need to agree on type with the bound parameter, not just with each other.
Related
Summary
The template-variable guardrail the translator emits for multi-select controls fails to compile whenever the bound control values are numeric. The panel renders a red error box instead of a chart.
Grafana community dashboard 9852 "node-exporter disk graphs" has a
CPUtemplate variable populated fromlabel_values(node_cpu_seconds_total, cpu), so its values are0and1. The migratedIO Wait per corepanel fails with:The emitted guard clause is:
When the selected option values all look numeric, Kibana binds
?CPUas an integer multi-value parameter. BothMV_CONTAINSarms then fail type-checking — one against the string literal".*", one against the keyword fieldlabels.cpu— and because this is a verification (compile-time) error, the whole query fails. TheMV_COUNT(?CPU) == 0short-circuit does not help; the statement never compiles.This is not specific to CPU indices. Any Grafana variable whose label values are numeric hits it: CPU/core IDs, ports, PIDs, HTTP status codes, queue or shard numbers, disk numbers.
Environment
feat/curated-dashboard-packs(PR feat!: curated dashboard packs, PromQL fidelity, and native-only dashboard artifacts #346) at5db742f9.5.0-SNAPSHOTnode_exporter, 15s scrapemetrics-node.prometheus-default,--field-profile prometheus_nativeReproduction
Then, in the uploaded dashboard:
CPUcontrol defaults to all options selected, including the synthetic.*entry. The panel renders, because the presence of.*keeps the parameter a keyword list..*, leaving only0, 1— a normal operator action, since.*is the tool's stand-in for Grafana's "All".IO Wait per corenow shows theverification_exceptionabove instead of a chart.Why the existing gates miss it
--validatereported13 queries: 13 passed, 0 failedfor this dashboard, and the pre-upload verification gate was8 Green / 5 Yellow / 0 Red. Validation binds the parameter differently from a live control, so the type conflict never appears offline. Only a browser session with a real control selection reproduces it — the interaction-audit case described inAGENTS.md.Suggested fix
Make the guardrail type-stable regardless of how Kibana infers the control parameter type. Options, roughly in order of preference:
TO_STRINGof the parameter elements against the keyword field, so a numeric binding cannot change the comparison type.MV_APPEND(".*", labels.cpu), which is keyword-typed at generation time but does not survive Kibana's re-inference of the selected values).Whatever the fix, the
.*sentinel and the field comparison need to agree on type with the bound parameter, not just with each other.Related