Skip to content

Range Grafana alerts migrate at the wrong resolution (hardcoded 1m step) #209

Description

@miguel-sanchez-elastic

User problem

When a Grafana alert is configured as Range — with a specific Interval (e.g. 1s) and time window — operators expect the migrated Kibana rule to evaluate metrics at the same resolution Grafana used.

Today, migrated range alerts always emit step=1m, regardless of what the operator configured in Grafana (Interval, Min step, Max data points). A rule evaluated every 1 second in Grafana can silently become a 1 minute rule in Kibana.

Why it matters

  • Wrong alert timing — breaches detected on a different cadence than the source rule
  • Loss of trust in migration — the PromQL expression looks correct, but behavior diverges
  • Hard to spot in review — reviewers focus on the expression and threshold, not hidden step defaults

Who is affected

Teams migrating Range Grafana Prometheus alerts where Interval ≠ 1m (common when Interval is 1s or Min step is tuned).

Not in scope for this ticket:

Current behavior

Grafana Range alert with Interval 1s, time range now-10m to now:

PROMQL index=metrics-prometheus-* step=1m value=(sum(node_cpu_seconds_total)) | WHERE value > 1.0

Expected behavior

Migrated query should reflect source resolution, e.g.:

PROMQL index=metrics-prometheus-* step=1s value=(sum(node_cpu_seconds_total)) | WHERE value > 1.0

The migrator should derive step= from exported rule metadata (intervalMs, Min step, Max data points / auto interval) and only fall back to a documented default when metadata is missing.

Success criteria

  • Range alert with Interval 1s / intervalMs: 1000 migrates with step=1s, not step=1m
  • Interval auto + known time range + max data points produces a step consistent with Grafana’s resolution (documented + tested)
  • Instant alerts (instant: true) never emit step= (regression guard; see Migrated alerts ignore the Instant query type and over-fire on stale data #200)
  • Migration output or comparison report notes when step was inferred vs taken directly from source
  • At least one end-to-end fixture: Range rule in Grafana → migrate → emitted step matches Interval

How to validate (QA)

  1. Create a Range Grafana alert, set Interval to 1s, threshold on a simple PromQL query.
  2. Migrate and inspect emitted ES|QL.
  3. Confirm step=1s (or equivalent), not step=1m.
  4. Repeat with Interval auto and a known 10m window; confirm step is plausible vs Grafana’s graph resolution.

Related

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

asset:alertsAlert, monitor, and rule migration assetsbugSomething isn't workingsource:grafanaGrafana source migrationworkstream:stabilityReliability, correctness, and quality work

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions