elastic / elastic/integrations

[redis_otel] "Keys per Database" dashboard panel always renders "Avg TTL (ms)" as null

Open
#19,437 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Handlebars
Stars
333
Forks
647
Avg merge
3d 4h
Merged PRs (30d)
209

Description

## Integration / environment

| | |
|---|---|
| Integration | `redis_otel` **0.2.0** (latest published) |
| Kibana / Elasticsearch | 9.4.2 |
| Metrics source | OpenTelemetry Collector `contrib`, `redisreceiver` **v0.151.0** |
| ES exporter | `elasticsearchexporter` with `mapping.mode: otel` |
| Dashboard | **[Redis OTel] Overview** (`redis_otel-overview`) |
| Panel | **Keys per Database** |

## Summary

On the **[Redis OTel] Overview** dashboard, the **Keys per Database** panel always renders the **Avg TTL (ms)** column as `null`, even though `redis.db.avg_ttl` is present and well-populated in the data. The **Keys** and **With TTL** columns render correctly.

## Root cause

The panel is an ES|QL Lens visualization with this query:

```esql
FROM metrics-redisreceiver.otel-*
| WHERE data_stream.dataset == "redisreceiver.otel"
| WHERE redis.db.keys IS NOT NULL
| STATS keys = MAX(redis.db.keys), expires = MAX(redis.db.expires), avg_ttl = AVG(redis.db.avg_ttl) BY db = attributes.db
| SORT keys DESC
| LIMIT 16
```

In `otel` mapping mode, the `elasticsearchexporter` writes **one `unit` per document**, so metrics with different units land in **different documents**:

- `redis.db.keys`, `redis.db.expires` → unit `{key}`
- `redis.db.avg_ttl` → unit `ms`

The clause `| WHERE redis.db.keys IS NOT NULL` therefore keeps **only** the `{key}` documents, in which `redis.db.avg_ttl` is null. `AVG(redis.db.avg_ttl)` over all-null rows returns `null`.

## Steps to reproduce

1. Install `redis_otel` 0.2.0; ship Redis metrics via the OTel `redisreceiver` to Elasticsearch through `elasticsearchexporter` with `mapping.mode: otel`.
2. Open **[Redis OTel] Overview → Keys per Database**.
3. **Avg TTL (ms)** is empty/null; **Keys** and **With TTL** show values.

## Evidence

Run against live data (single db, `attributes.db = "0"`):

| Query | keys | expires | avg_ttl |
|---|---|---|---|
| As shipped (`WHERE redis.db.keys IS NOT NULL`) | 5068 | 5066 | **null** |
| Filter on the dimension instead (`WHERE attributes.db IS NOT NULL`) | 5068 | 5066 | **391,577,632** |

Document split by unit (same dataset, `attributes.db` present):

```
unit={key} docs=7765 has keys=7765 has expires=7765 has avg_ttl=0
unit=ms docs=7765 has keys=0 has expires=0 has avg_ttl=7765
# 0 documents contain both redis.db.keys and redis.db.avg_ttl
```

## Proposed fix

Filter on the grouping dimension (`attributes.db`), which is present on **all** per-db metric documents, instead of on a unit-specific metric field:

```diff
FROM metrics-redisreceiver.otel-*
| WHERE data_stream.dataset == "redisreceiver.otel"
- | WHERE redis.db.keys IS NOT NULL
+ | WHERE attributes.db IS NOT NULL
| STATS keys = MAX(redis.db.keys), expires = MAX(redis.db.expires), avg_ttl = AVG(redis.db.avg_ttl) BY db = attributes.db
| SORT keys DESC
| LIMIT 16
```

`MAX`/`AVG` already ignore nulls per metric, so **Keys**, **With TTL** and **Avg TTL (ms)** all populate correctly.

Dashboard source: `packages/redis_otel/kibana/dashboard/` (object `redis_otel-overview`, panel "Keys per Database").

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.