open-telemetry / open-telemetry/opentelemetry-python-contrib
SDK Exporter for AWS CloudWatch Embedded Metrics Format (EMF)
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.1k
- Forks
- 1.1k
- Avg merge
- 4d 15h
- Merged PRs (30d)
- 16
Description
Before opening a feature request against this repo, consider whether the feature should/could be implemented in the other OpenTelemetry client libraries. If so, please open an issue on opentelemetry-specification first.
consider whether the feature should/could be implemented in the other OpenTelemetry client libraries
It looks like the Zipkin and OTLP metric exporters are in the OTel SDK codebase, so I could see a case for this belonging there, but on the other hand, other AWS-related extensions have been added to this codebase.
Is your feature request related to a problem?
I mention this further down, but TL;DR, it'd be nice to be able to ship metrics and logs to CloudWatch via stdout, e.g. via the awslogs exporter built into the docker daemon.
This can save compute cost in AWS Lambda and AWS ECS (for example) by not having to turn on the OpenTelemetry Extension for Lambda or running an additional ADOT instance with each task (or a single shared instance at a discoverable URL).
Describe the solution you'd like
Forgive me if this already exists and I just couldn't find it.
Would it be possible or desirable to implement an additional metrics exporter (not as part of the collector but as part of the OTel SDK) that ships metrics to the console but in the AWS CloudWatch Embedded Metrics format?
What I imagine is something like:
pip install opentelemetry-exporter-emf
OTEL_METRICS_EXPORTER=emf \
OTEL_SERVICE_NAME=otel-playground-app \
opentelemetry-instrument -- python script_that_produces_metrics.py
# some logs would now show up in stdout in the emf format
Which would print metrics in batches to stdout as EMF-formatted JSON.
Describe alternatives you've considered
I realize the OpenTelemetry collector already can export in this format.
I imagined having an EMF exporter built-into the SDK would be nice in environments where you don't want to spend additional money/resources on running a dedicated collector, and relying on systems that collect logs from stdout, e.g. docker's awslogs driver forwarding logs and metrics directly to CloudWatch.
Which alternative solutions or features have you considered?
I couldn't think of any alternatives that avoid running a dedicated collector 😅.
Additional context
Add any other context about the feature request here.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Read the OpenTelemetry specification guidance and the existing Collector EMF exporter first, then determine whether an SDK exporter belongs in this repository. Done means an agreed ownership and scope plus a Python package and configuration that emit batched EMF JSON to stdout for the described Lambda and ECS use cases.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python
- Domain
- observability
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100