open-telemetry / open-telemetry/opentelemetry-python
Opentelemetry prometheus exported datadog parser error
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2.6k
- Forks
- 1k
- Avg merge
- 4d 15h
- Merged PRs (30d)
- 19
Description
Trying to expose a prometheus endpoint so that datadog can pull the custom metrics from that endpoint. For this I am using a envvironment variable OTEL_METRICS_EXPORTER=prometheus, which opens up a port at 9464 to scrap metrics.
When datadog parse the response from the endpoint :9464/metrics, the datadog agent throws exception related to parsing.
Steps to reproduce
- Create a Java project using opentelemetry dependencies and create a guage
- Try to record some metrics in the gauge.
- Add env variable called OTEL_METRICS_EXPORTER=prometheus
- Start the program
- Configure Datadog agent to scrap from the prometheus endpoint i.e. :9464/metrics
- Datadog agent should throw parsing exception
What is the expected behavior?
The custom metrics should reach to the datadog metrics's dashboard
What is the actual behavior?
Parsing error thrown by datadog agent
Additional context
Attaching
datadog_error.txt
prometheus_exported_response.txt
prometheus exported response and the parsing error from datadog.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by comparing prometheus_exported_response.txt with datadog_error.txt and review the Prometheus exporter behavior enabled by OTEL_METRICS_EXPORTER=prometheus. Reproduce the gauge export and Datadog scrape using the six listed steps. Done means the exported response is accepted by the Datadog agent and the custom metric reaches its dashboard.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- prometheus, python
- Domain
- observability
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100