elastic / elastic/observability-migration-platform
[Datadog] Rate/counter temporality & unit correctness on Agent to OTel collector switch
- Dominant language
- Python
- Stars
- 6
- Forks
- 8
- Avg merge
- 2d 22h
- Merged PRs (30d)
- 23
Description
## Context
The Datadog ES|QL translator has a counter-aware path (native TS|QL `RATE()`/`INCREASE()` when live `_field_caps` prove `time_series_metric_kind == counter` or a `counter_*` type) and otherwise emits a bucket-span division rate approximation. `metric_map` correctly classifies Class-2 entries (`transform` / `attribute_filter` / non-1 `unit_scale`) as gaps rather than silently applying wrong math.
Remaining risk: when an operator switches data collection from the Datadog Agent to an OTel collector, the same logical metric can change **temporality** (delta vs cumulative), **unit** (e.g. cores vs nanocores, bytes vs KiB), and counter typing. Without field-caps proof, the rate approximation can be numerically wrong, and unit changes are not auto-reconciled.
## Ask
- Document the temporality/unit hazards for Agent to OTel switches.
- Consider surfacing a preflight warning when a mapped metric's target unit/temporality likely differs from source.
- Evaluate `unit_scale` ergonomics for the common conversions.
## Acceptance
- Preflight or report warns on likely temporality/unit mismatch for rate/counter metrics.
- Docs describe the hazard and the `--metric-map-file` `unit_scale`/`transform` remedy.
Correctness item; distinct from the (stale) roadmap text that claimed rates were unhandled.
Contributor guide
Research direction
Read the Datadog ES|QL translator's metric_map handling and the --metric-map-file documentation, then trace how _field_caps establishes counter typing and how unit_scale/transform are reported. Done means a preflight or report warns about likely temporality or unit mismatches for rate/counter metrics, and the documentation explains the hazard and remedies.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- observability-sre
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100