opensearch-project / opensearch-project/data-prepper

[BUG] The data schema named `Data Prepper` in the `observability->traces->services` page does not display services properly

Open
#5,116 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug help wanted
Dominant language
Java
Stars
374
Forks
355
Avg merge
3d 18h
Merged PRs (30d)
8

Description

Describe the bug
A clear and concise description of what the bug is.

To Reproduce
Steps to reproduce the behavior:

  1. Execute the following command on the Ubuntu 22.04.5 instance with SDKMAN! and Docker Engine installed. Refer to https://github.com/linghengqian/data-prepper-logs-test . This uses OpenJDK 23.
sdk install java 23-open
sdk use java 23-open

git clone git@github.com:linghengqian/data-prepper-logs-test.git
cd ./data-prepper-logs-test/

docker compose --file ./opensearch/docker-compose.yml up -d

./mvnw clean dependency:get -Dartifact=io.opentelemetry.javaagent:opentelemetry-javaagent:2.9.0

./mvnw clean spring-boot:run \
  -Dspring-boot.run.agents="$HOME/.m2/repository/io/opentelemetry/javaagent/opentelemetry-javaagent/2.9.0/opentelemetry-javaagent-2.9.0.jar" \
  -Dspring-boot.run.jvmArguments="\
  -Dotel.service.name='1-linghengqian-smoke-tests' \
  -Dotel.exporter.otlp.endpoint='http://localhost:24321'\
  "

This occupies host ports 14321 and 24321.

  1. Open http://localhost:14321/app/observability-traces#/services in Microsoft Edge browser and log in with the
    account admin and the password opensearchNode1Test.
    The data schema named Data Prepper will not have any services. Although traces do collect APM data.
  • image
  • image
  1. However, the data schema named Custom source will display 1-linghengqian-smoke-tests normally. From beginning to end, the Spring Boot application did not throw any exceptions.
  • image

Expected behavior
A clear and concise description of what you expected to happen.

  • The data schema named Data Prepper in the observability->traces->services page can display services normally.

Screenshots
If applicable, add screenshots to help explain your problem.

  • Already mentioned above.

Environment (please complete the following information):

  • OS: [e.g. Ubuntu 20.04 LTS] Ubuntu WSL 22.04.5 LTS
  • Version [e.g. 22] opensearchproject/data-prepper:2.10.1

Additional context
Add any other context about the problem here.

  • An interesting point is that neither https://github.com/opensearch-project/data-prepper/tree/main/examples/jaeger-hotrod nor https://github.com/opensearch-project/data-prepper/pull/4972 have this problem. Am I supposed to understand that for a data schema named Data Prepper, the service map is only available when there are multiple services?
  • Also, the data schema named Data Prepper seems to be able to directly accept the grpc data sent by the otlp exporter, so why does the OpenSearch website require the additional deployment of data prepper? otel/opentelemetry-collector-contrib can actually directly convert the Opentelemetry SDK data into opensearch documents. Another additional topic is, wouldn’t it be better if the opensearch dashboard directly integrates the logs, metrics, and traces sent by the Opentelemetry SDK into a single web page? Why split logs, metrics, and traces into different web pages?
  • image

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the issue with the linked data-prepper-logs-test repository and the observability-traces#/services page using the stated OpenJDK, Docker, and Data Prepper versions. Compare the Data Prepper schema with the Custom source schema and the referenced examples; done means the Data Prepper schema displays the collected service normally.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, java
Domain
observability-sre
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.