influxdata / influxdata/telegraf

Support for exposing container memory reservation in ECS

Open
#12,271 5 comments 1 reaction 0 assignees View on GitHub
feature request
Dominant language
Go
Stars
17.8k
Forks
5.8k
Avg merge
1d 20h
Merged PRs (30d)
161

Description

### Use Case

Currently the ecs input plugin exposes memory usage data and also the container/task memory limit. These are exposed via the docker stats endpoint.
It would be extremely useful to also know about the memory reservation (soft limit) and (optional) CPU reservation associated with each container, as this is one of the primary drivers for resource allocation within ECS and can help detect over/underprovisioning.
As far as I can tell this information is not available from the ECS task metadata, but could be parsed from the task definition on startup, given appropriate IAM permissions. Since task definitions are immutable, this would be a one-off operation.
The soft limit is also available from 'docker inspect' under the `MemoryReservation` value although I don't think that's easily accessible from within the task.

### Expected behavior

An `ecs_container_mem_reservation` metric with the value in bytes of the container reservation, and an `ecs_container_cpu_reservation` metric with the number of CPU units reserved for the container (value of 0 if no reservation is set?)

### Actual behavior

This data is not exposed currently

### Additional info

I'm going to raise a feature request with AWS to request that these values are exposed in the container stats endpoint. That would obviously be the best solution, although if they do add it, it may require container metadata v4 support in telegraf.

Contributor guide

Open the contributing guide

Research direction

Start with the ECS input plugin and its existing Docker stats metrics, then review how task definitions and ECS metadata are accessed. Done means exposing container memory reservation in bytes and CPU reservation in CPU units, including the specified zero value when no CPU reservation is set.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, go
Domain
cloud
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.