open-telemetry / open-telemetry/opentelemetry-python-contrib

SDK Exporter for AWS CloudWatch Embedded Metrics Format (EMF)

Open
#2,490 1 comment 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

feature-request
Dominant language
Python
Stars
1.1k
Forks
1.1k
Avg merge
4d 15h
Merged PRs (30d)
16

Description

Before opening a feature request against this repo, consider whether the feature should/could be implemented in the other OpenTelemetry client libraries. If so, please open an issue on opentelemetry-specification first.

consider whether the feature should/could be implemented in the other OpenTelemetry client libraries

It looks like the Zipkin and OTLP metric exporters are in the OTel SDK codebase, so I could see a case for this belonging there, but on the other hand, other AWS-related extensions have been added to this codebase.

Is your feature request related to a problem?

I mention this further down, but TL;DR, it'd be nice to be able to ship metrics and logs to CloudWatch via stdout, e.g. via the awslogs exporter built into the docker daemon.

This can save compute cost in AWS Lambda and AWS ECS (for example) by not having to turn on the OpenTelemetry Extension for Lambda or running an additional ADOT instance with each task (or a single shared instance at a discoverable URL).

Describe the solution you'd like

Forgive me if this already exists and I just couldn't find it.

Would it be possible or desirable to implement an additional metrics exporter (not as part of the collector but as part of the OTel SDK) that ships metrics to the console but in the AWS CloudWatch Embedded Metrics format?

What I imagine is something like:

pip install opentelemetry-exporter-emf

OTEL_METRICS_EXPORTER=emf \
OTEL_SERVICE_NAME=otel-playground-app \
opentelemetry-instrument -- python script_that_produces_metrics.py

# some logs would now show up in stdout in the emf format

Which would print metrics in batches to stdout as EMF-formatted JSON.

Describe alternatives you've considered

I realize the OpenTelemetry collector already can export in this format.

I imagined having an EMF exporter built-into the SDK would be nice in environments where you don't want to spend additional money/resources on running a dedicated collector, and relying on systems that collect logs from stdout, e.g. docker's awslogs driver forwarding logs and metrics directly to CloudWatch.

Which alternative solutions or features have you considered?

I couldn't think of any alternatives that avoid running a dedicated collector 😅.

Additional context

Add any other context about the feature request here.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read the OpenTelemetry specification guidance and the existing Collector EMF exporter first, then determine whether an SDK exporter belongs in this repository. Done means an agreed ownership and scope plus a Python package and configuration that emit batched EMF JSON to stdout for the described Lambda and ECS use cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, python
Domain
observability
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.