elastic / elastic/observability-migration-platform

Grafana: PromQL vector-to-vector comparison (`A == B`) emits a boolean for every series instead of filtering to matching series

Open
#375 0 comments 0 reactions 0 assignees View on GitHub
asset:dashboards bug program:grafana-engine source:grafana
Dominant language
Python
Stars
6
Forks
8
Avg merge
2d 22h
Merged PRs (30d)
23

Description

## Summary

In PromQL, a comparison between two vectors without the `bool` modifier is a **filter**: it
returns the left-hand series *with its original value* for the elements where the comparison
holds, and drops every non-matching series. The migration instead emits ES|QL arithmetic that
evaluates the comparison to a **boolean for every series**, so the panel shows all series with
`true`/`false` instead of only the matching ones with their real values.

## Reproduction

Dashboard: [grafana.com 13332 "kube-state-metrics-v2"](https://grafana.com/grafana/dashboards/13332-kube-state-metrics-v2/),
panels `current==max` and `current==min`.

Source PromQL:

```promql
kube_hpa_status_current_replicas{hpa=~".*"} == kube_hpa_spec_max_replicas{hpa=~".*"}
```

Seed data — three HPAs, only one of which sits at its max:

| hpa | current | min | max |
|---|---|---|---|
| api-gateway | 3 | 2 | 10 |
| order-service | **5** | 1 | **5** |
| nginx-web | 2 | 2 | 6 |

## Evidence

**Grafana** renders exactly one series, `order-service`, with value **5**.

**Migrated Kibana** renders three series with boolean values:

```
columns: ['time_bucket', 'labels.hpa', 'computed_value']
['2026-08-21T09:56:00.000Z', 'api-gateway', False]
['2026-08-21T09:56:00.000Z', 'nginx-web', False]
['2026-08-21T09:56:00.000Z', 'order-service', True]
```

Emitted ES|QL:

```
TS metrics-k8s.prometheus-parity
| WHERE metrics.kube_hpa_status_current_replicas IS NOT NULL OR metrics.kube_hpa_spec_max_replicas IS NOT NULL
| STATS kube_hpa_status_current_replicas_hpa = AVG(LAST_OVER_TIME(metrics.kube_hpa_status_current_replicas)),
kube_hpa_spec_max_replicas_hpa = AVG(LAST_OVER_TIME(metrics.kube_hpa_spec_max_replicas))
BY time_bucket = TBUCKET(75, ?_tstart, ?_tend), labels.hpa
| EVAL computed_value = (kube_hpa_status_current_replicas_hpa == kube_hpa_spec_max_replicas_hpa)
| DROP kube_hpa_status_current_replicas_hpa, kube_hpa_spec_max_replicas_hpa
| SORT time_bucket ASC
```

Two separate losses in one expression:

1. **Filtering is lost** — non-matching series are returned as `false` instead of being dropped.
2. **The value is lost** — the matching series reports `true` rather than the left operand's
value (`5`).

For contrast, the *scalar* comparison form is handled correctly elsewhere in the same
dashboard — `sum(kube_persistentvolumeclaim_status_phase{phase="Bound"}==1)` emits
`| WHERE metrics.kube_persistentvolumeclaim_status_phase == 1` and matches Grafana (2 vs 2).
So the gap is specific to **vector-to-vector** comparison.

## Expected

`A == B` (no `bool`) should keep filter semantics, e.g.:

```
| EVAL computed_value = CASE(a == b, a, NULL)
```

which both drops the non-matching series (NULL renders as no line) and preserves the left
operand's value. The same applies to `!=`, `>`, `<`, `>=`, `<=` between two vectors.
Where that cannot be expressed faithfully, the panel should degrade with an explicit reason
rather than silently changing the semantics.

## Secondary: misleading warning text

Both panels are reported as `migrated_with_warnings` with:

```
"Approximated PromQL arithmetic using same-bucket ES|QL math"
"Dropped variable-driven label filters during migration"
```

The second reason is inaccurate — the expression contains no dashboard variables. The only
label filter present is the literal `hpa=~".*"`, which is indeed dropped from the emitted
ES|QL (harmless here since `.*` matches everything, but the message points an operator at the
wrong cause).

Contributor guide

Open the contributing guide

Research direction

Reproduce the Grafana dashboard 13332 panels `current==max` and `current==min`, then trace the vector-to-vector comparison translation alongside the existing scalar comparison path. Verify that non-matching series are omitted, matching series retain the left operand value, and misleading warning text is corrected or an explicit degradation reason is emitted.

Written by the indexing model from the issue text.

Assessment

Tech stack
elasticsearch, grafana, python
Domain
observability-sre, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.