opensearch-project / opensearch-project/data-prepper
Improved otel_metrics_source documentation
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 374
- Forks
- 354
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 8
Description
Is your feature request related to a problem? Please describe.
In Data Prepper v1.5.1 the otel_metrics_source cannot be used with the string_converter plugin.
Describe the solution you'd like
Documentation on plugin/feature compatibility.
Describe alternatives you've considered (Optional)
Enhance pipeline validation on startup to detect incomatibility issues.
Additional context
Related community forum question
pipelines.yaml
entry-pipeline:
source:
otel_metrics_source:
ssl: false
sink:
- stdout:
otel-collector-config.yaml
receivers:
prometheus:
config:
scrape_configs:
- job_name: 'data-prepper'
scrape_interval: 10s
metrics_path: "/metrics/prometheus"
static_configs:
- targets: ['host.docker.internal:4900']
exporters:
otlp/2:
endpoint: data-prepper:21891
tls:
insecure: true
service:
pipelines:
metrics:
receivers: [prometheus]
exporters: [otlp/2]
prometheus.yaml
docker-compose.yaml
version: "3.9"
services:
data-prepper:
restart: unless-stopped
image: opensearch-data-prepper:2.0.0-SNAPSHOT
working_dir: /usr/share/data-prepper/
volumes:
- ./data-prepper/config/data-prepper-config.yaml:/usr/share/data-prepper/config/data-prepper-config.yaml
- ./data-prepper/config/pipelines.yaml:/usr/share/data-prepper/pipelines.yaml
- ../../shared-config/log4j2.properties:/usr/share/data-prepper/config/log4j.properties
ports:
- "2021:2021"
otel-collector:
restart: unless-stopped
image: otel/opentelemetry-collector:0.40.0
command: ["--config=/etc/otel-collector-config.yml"]
volumes:
- ./opentelemetry-collector/otel-collector-config.yml:/etc/otel-collector-config.yml
depends_on:
- data-prepper
ports:
- "4317:4317"
prometheus:
restart: unless-stopped
image: prom/prometheus
volumes:
- ./prometheus/prometheus.yaml:/etc/prometheus/prometheus.yaml
ports:
- "9000:9000"
depends_on:
- otel-collector
I recommend using the smoke-tests docker-compose.yaml as a base for setup.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the smoke-tests docker-compose.yaml and the example pipelines.yaml, including the otel_metrics_source and string_converter configuration described here. Check the existing plugin documentation for compatibility guidance. Done means documenting the incompatibility clearly, with the relevant configuration context and any supported alternatives.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker-compose
- Domain
- documentation, observability
- Issue type
- Documentation
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100