elastic / elastic/opentelemetry
Default OpenTelemetry Collector traces pipeline to mOTLP instead of Elasticsearch in Elastic Cloud Hosted
- Dominant language
- Python
- Stars
- 37
- Forks
- 50
- Avg merge
- 23h 31m
- Merged PRs (30d)
- 3
Description
### Summary
Currently, when onboarding an OpenTelemetry Collector via OpAmp in Elastic Cloud Hosted (9.4.2), the default configuration proposes sending traces directly to Elasticsearch. This appears to contradict recommended practices and documentation, which suggest traces should instead go to mOTLP or an EDOT Collector/Gateway for proper aggregation and visibility in the APM application.
### Example Configuration
```
receivers:
otlp:
protocols:
grpc:
endpoint: "0.0.0.0:4317"
exporters:
elasticsearch/otel:
endpoints:
- ""
api_key: "${API_KEY}"
mapping:
mode: "otel"
otlp:
endpoint: ""
tls:
insecure: true
service:
pipelines:
traces:
receivers:
- "otlp"
exporters:
- "elasticsearch/otel"
```
### Expected Behavior
- The default should be to send traces to mOTLP.
- If mOTLP is not available (e.g., on-prem/self-managed installs), traces should be routed to an EDOT Gateway, as per documentation:
- https://www.elastic.co/docs/reference/edot-collector/modes#gateway-requirements-for-self-managed-environments
- https://www.elastic.co/docs/reference/opentelemetry/architecture/hosts_vms#deployment-scenarios
- https://www.elastic.co/docs/reference/opentelemetry/architecture#vendor-neutral-collector-deployments-on-edge-environments
### Additional Notes
- Current config may overwhelm Elasticsearch with high trace volumes.
- Related documentation may also need clarification: https://www.elastic.co/docs/reference/fleet/add-otel-collector#verify-the-collector-connection
### Action Items
- Update default configuration for traces in Elastic Cloud Hosted to route through mOTLP.
- Update documentation to clarify recommended routing for both Elastic Cloud and on-prem/self-managed installs.
/cc @fabri and team for visibility.
[View original Slack conversation](https://elastic.slack.com/archives/C031284ANNL/p1780557531713779?thread_ts=1780557531.713779&cid=C031284ANNL)
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reproducing the OpenTelemetry Collector onboarding flow via OpAmp in Elastic Cloud Hosted 9.4.2 and inspect the generated traces pipeline. Compare the current Elasticsearch routing with the linked mOTLP and EDOT deployment guidance. Done means Hosted defaults to mOTLP, self-managed guidance covers EDOT Gateway routing, and the collector verification documentation is clear.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- elasticsearch
- Domain
- documentation, observability-sre
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100