elastic / elastic/integrations
[redis_otel] "Keys per Database" dashboard panel always renders "Avg TTL (ms)" as null
- Dominant language
- Handlebars
- Stars
- 333
- Forks
- 647
- Avg merge
- 3d 4h
- Merged PRs (30d)
- 209
Description
## Integration / environment
| | |
|---|---|
| Integration | `redis_otel` **0.2.0** (latest published) |
| Kibana / Elasticsearch | 9.4.2 |
| Metrics source | OpenTelemetry Collector `contrib`, `redisreceiver` **v0.151.0** |
| ES exporter | `elasticsearchexporter` with `mapping.mode: otel` |
| Dashboard | **[Redis OTel] Overview** (`redis_otel-overview`) |
| Panel | **Keys per Database** |
## Summary
On the **[Redis OTel] Overview** dashboard, the **Keys per Database** panel always renders the **Avg TTL (ms)** column as `null`, even though `redis.db.avg_ttl` is present and well-populated in the data. The **Keys** and **With TTL** columns render correctly.
## Root cause
The panel is an ES|QL Lens visualization with this query:
```esql
FROM metrics-redisreceiver.otel-*
| WHERE data_stream.dataset == "redisreceiver.otel"
| WHERE redis.db.keys IS NOT NULL
| STATS keys = MAX(redis.db.keys), expires = MAX(redis.db.expires), avg_ttl = AVG(redis.db.avg_ttl) BY db = attributes.db
| SORT keys DESC
| LIMIT 16
```
In `otel` mapping mode, the `elasticsearchexporter` writes **one `unit` per document**, so metrics with different units land in **different documents**:
- `redis.db.keys`, `redis.db.expires` → unit `{key}`
- `redis.db.avg_ttl` → unit `ms`
The clause `| WHERE redis.db.keys IS NOT NULL` therefore keeps **only** the `{key}` documents, in which `redis.db.avg_ttl` is null. `AVG(redis.db.avg_ttl)` over all-null rows returns `null`.
## Steps to reproduce
1. Install `redis_otel` 0.2.0; ship Redis metrics via the OTel `redisreceiver` to Elasticsearch through `elasticsearchexporter` with `mapping.mode: otel`.
2. Open **[Redis OTel] Overview → Keys per Database**.
3. **Avg TTL (ms)** is empty/null; **Keys** and **With TTL** show values.
## Evidence
Run against live data (single db, `attributes.db = "0"`):
| Query | keys | expires | avg_ttl |
|---|---|---|---|
| As shipped (`WHERE redis.db.keys IS NOT NULL`) | 5068 | 5066 | **null** |
| Filter on the dimension instead (`WHERE attributes.db IS NOT NULL`) | 5068 | 5066 | **391,577,632** |
Document split by unit (same dataset, `attributes.db` present):
```
unit={key} docs=7765 has keys=7765 has expires=7765 has avg_ttl=0
unit=ms docs=7765 has keys=0 has expires=0 has avg_ttl=7765
# 0 documents contain both redis.db.keys and redis.db.avg_ttl
```
## Proposed fix
Filter on the grouping dimension (`attributes.db`), which is present on **all** per-db metric documents, instead of on a unit-specific metric field:
```diff
FROM metrics-redisreceiver.otel-*
| WHERE data_stream.dataset == "redisreceiver.otel"
- | WHERE redis.db.keys IS NOT NULL
+ | WHERE attributes.db IS NOT NULL
| STATS keys = MAX(redis.db.keys), expires = MAX(redis.db.expires), avg_ttl = AVG(redis.db.avg_ttl) BY db = attributes.db
| SORT keys DESC
| LIMIT 16
```
`MAX`/`AVG` already ignore nulls per metric, so **Keys**, **With TTL** and **Avg TTL (ms)** all populate correctly.
Dashboard source: `packages/redis_otel/kibana/dashboard/` (object `redis_otel-overview`, panel "Keys per Database").
Contributor guide
Assessment
This issue has not been assessed yet.