[BUG] Pipeline Health dashboard shows blank Cortex and Data Prepper panels with Docker Compose

What is the bug?

With the default Docker Compose deployment, the Observability Pipeline Health dashboard shows no data for the Cortex and Data Prepper panels even while the stack is healthy and telemetry is being ingested.

The OTel Collector memory panel is also empty.

The underlying metrics endpoints expose data, but the Docker Compose OTel Collector configuration only scrapes its own metrics endpoint and Envoy. It does not scrape Cortex or Data Prepper before remote-writing metrics to Cortex.

The memory panel also queries otelcol_process_memory_rss_bytes, while the metric emitted by the collector is otelcol_process_memory_rss.

How can one reproduce the bug?

1. Start the default stack with fresh volumes:

   docker compose down -v
   docker compose up -d
   

2. Start the example canary to generate telemetry:

   docker compose up -d example-canary
   

3. Wait for several canary invocations.

4. Open OpenSearch Dashboards and navigate to:

   Observability Stack → Dashboards → Observability Pipeline Health
   

5. Observe that the following panels show no results:

   • Cortex Ingestion Rate
   • Cortex Active Time Series
   • Cortex Query Latency
   • Data Prepper records processed
   • Data Prepper pipeline latency
   • Data Prepper documents written
   • Data Prepper buffer metrics
   • OTel Collector Memory

6. Verify that the source metrics exist:

   curl -s http://localhost:9090/metrics |
     grep '^cortex_ingester_active_series'
   

   docker compose exec data-prepper \
     curl -s http://localhost:4900/metrics/prometheus |
     grep 'otel_traces_pipeline_recordsProcessed_total'
   

7. Query Cortex directly:

   curl -sG 'http://localhost:9090/prometheus/api/v1/query' \
     --data-urlencode 'query=cortex_ingester_active_series'
   

   The direct metrics endpoint contains the metric, but the Cortex query returns no series.

What is the expected behavior?

The Pipeline Health dashboard should display Cortex and Data Prepper self-monitoring metrics when running the default Docker Compose stack.

The OTel Collector memory panel should query the metric actually emitted by the collector.

What is your host/environment?

• Host OS: macOS 26.5.1, arm64
• Docker CLI: 29.4.0
• Docker Compose: v5.1.2
• Repository commit: 6e5b56e
• OpenSearch image: opensearchproject/opensearch:3.8.0
• Data Prepper image: opensearchproject/data-prepper:2.16.0
• OTel Collector image: otel/opentelemetry-collector-contrib:0.156.0

Are there any screenshots?

Screenshots can be provided showing:

• working OTel Collector throughput panels
• blank Cortex and Data Prepper panels
• blank OTel Collector memory panel

Do you have any additional context?

The Docker Compose collector currently configures the metrics pipeline as:

metrics:
  receivers: [otlp, prometheus/self, prometheus/envoy]

There are no receivers for Cortex or Data Prepper.

As a causal test, I ran a temporary OTel Collector configuration that scraped:

• prometheus:9090/metrics
• data-prepper:4900/metrics/prometheus

and remote-wrote those metrics to Cortex. After that change, the same dashboard PromQL queries returned values for Cortex and Data Prepper.

The dashboard also uses:

otelcol_process_memory_rss_bytes

but the emitted metric is:

otelcol_process_memory_rss

The Helm collector configuration already contains Cortex and Data Prepper scrape receivers, which indicates a configuration parity gap between Docker Compose and Helm.

Start with the Docker Compose OTel Collector metrics pipeline and compare it with the Helm collector configuration, which already includes Cortex and Data Prepper scrape receivers. Reproduce the issue with the documented docker compose commands, then inspect the Pipeline Health dashboard queries, especially the OTel Collector memory query. Done means the dashboard displays Cortex, Data Prepper, and collector memory metrics in the default Compose stack.