elastic / elastic/observability-migration-platform

[Datadog] Rate/counter temporality & unit correctness on Agent to OTel collector switch

Open
#308 0 comments 0 reactions 0 assignees View on GitHub
program:datadog quality source:datadog
Dominant language
Python
Stars
6
Forks
8
Avg merge
2d 22h
Merged PRs (30d)
23

Description

## Context
The Datadog ES|QL translator has a counter-aware path (native TS|QL `RATE()`/`INCREASE()` when live `_field_caps` prove `time_series_metric_kind == counter` or a `counter_*` type) and otherwise emits a bucket-span division rate approximation. `metric_map` correctly classifies Class-2 entries (`transform` / `attribute_filter` / non-1 `unit_scale`) as gaps rather than silently applying wrong math.

Remaining risk: when an operator switches data collection from the Datadog Agent to an OTel collector, the same logical metric can change **temporality** (delta vs cumulative), **unit** (e.g. cores vs nanocores, bytes vs KiB), and counter typing. Without field-caps proof, the rate approximation can be numerically wrong, and unit changes are not auto-reconciled.

## Ask
- Document the temporality/unit hazards for Agent to OTel switches.
- Consider surfacing a preflight warning when a mapped metric's target unit/temporality likely differs from source.
- Evaluate `unit_scale` ergonomics for the common conversions.

## Acceptance
- Preflight or report warns on likely temporality/unit mismatch for rate/counter metrics.
- Docs describe the hazard and the `--metric-map-file` `unit_scale`/`transform` remedy.

Correctness item; distinct from the (stale) roadmap text that claimed rates were unhandled.

Contributor guide

Open the contributing guide

Research direction

Read the Datadog ES|QL translator's metric_map handling and the --metric-map-file documentation, then trace how _field_caps establishes counter typing and how unit_scale/transform are reported. Done means a preflight or report warns about likely temporality or unit mismatches for rate/counter metrics, and the documentation explains the hazard and remedies.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
observability-sre
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.