Monitoring indices map cgroup memory metrics as keyword instead of numeric
- Dominant language
- Go
- Stars
- 12.7k
- Forks
- 5k
- Avg merge
- 2d 15m
- Merged PRs (30d)
- 385
Description
Several cgroup memory metrics collected by Elastic Cloud Enterprise (ECE) 4.0.3 monitoring are indexed as `keyword` fields instead of numeric types.
As a result, these metrics cannot be used for:
* Lens visualizations
* TSVB
* ES|QL numeric operations
* aggregations
* sorting
* threshold-based alerting
This makes it difficult to monitor container memory usage and memory limits, despite the values themselves being numeric.
## Affected fields
Examples include:
* `memory.limit.bytes`
* `memory.usage.bytes`
(There may be additional cgroup memory metrics affected.)
## Current mapping
Example:
```json
"memory": {
"properties": {
"limit": {
"properties": {
"bytes": {
"type": "keyword"
}
}
},
"usage": {
"properties": {
"bytes": {
"type": "keyword"
}
}
}
}
}
```
## Expected mapping
These fields should be mapped as numeric values, for example:
```json
"type": "long"
```
since they represent byte counts.
## Why this matters
Memory limit and usage are fundamental infrastructure metrics that operators commonly use to:
* monitor memory consumption over time
* calculate memory utilization percentages
* create threshold alerts
* build dashboards
* correlate memory pressure with CPU throttling and other container metrics
When stored as `keyword`, these common observability workflows are unnecessarily limited.
## Reproduction
1. Enable ECE monitoring.
2. Inspect the mapping of the monitoring indices.
3. Observe that cgroup memory byte fields are mapped as `keyword`.
4. Attempt to build a Lens visualization or perform numeric aggregations on these fields.
## Additional context
A discussion was started on the Elastic forums where it was suggested that this should be tracked as a GitHub issue.
Forum discussion:
[https://discuss.elastic.co/t/ece-monitoring-cgroup-memory-fields-mapped-as-keyword-instead-of-numeric/386861](https://discuss.elastic.co/t/ece-monitoring-cgroup-memory-fields-mapped-as-keyword-instead-of-numeric/386861)
## Expected outcome
Map cgroup memory byte metrics using an appropriate numeric type (e.g. `long`) so they can be used consistently throughout the Elastic Stack for visualization, alerting, ES|QL, and analytics.
Contributor guide
Research direction
Start by locating the monitoring index mapping responsible for the cgroup fields named in the issue, especially memory.limit.bytes and memory.usage.bytes. Inspect how their types are assigned, then verify the affected mappings use a numeric type and that the monitoring index supports numeric aggregations and sorting.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- observability-sre
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100