opensearch-project / opensearch-project/data-prepper
[BUG] The data schema named `Data Prepper` in the `observability->traces->services` page does not display services properly
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 374
- Forks
- 355
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 8
Description
Describe the bug
A clear and concise description of what the bug is.
- The data schema named
Data Prepperin theobservability->traces->servicespage does not display services properly. I created a minimal reproducible example at https://github.com/linghengqian/data-prepper-logs-test .
To Reproduce
Steps to reproduce the behavior:
- Execute the following command on the Ubuntu 22.04.5 instance with
SDKMAN!andDocker Engineinstalled. Refer to https://github.com/linghengqian/data-prepper-logs-test . This uses OpenJDK 23.
sdk install java 23-open
sdk use java 23-open
git clone git@github.com:linghengqian/data-prepper-logs-test.git
cd ./data-prepper-logs-test/
docker compose --file ./opensearch/docker-compose.yml up -d
./mvnw clean dependency:get -Dartifact=io.opentelemetry.javaagent:opentelemetry-javaagent:2.9.0
./mvnw clean spring-boot:run \
-Dspring-boot.run.agents="$HOME/.m2/repository/io/opentelemetry/javaagent/opentelemetry-javaagent/2.9.0/opentelemetry-javaagent-2.9.0.jar" \
-Dspring-boot.run.jvmArguments="\
-Dotel.service.name='1-linghengqian-smoke-tests' \
-Dotel.exporter.otlp.endpoint='http://localhost:24321'\
"
This occupies host ports 14321 and 24321.
- Open
http://localhost:14321/app/observability-traces#/servicesin Microsoft Edge browser and log in with the
accountadminand the passwordopensearchNode1Test.
The data schema namedData Prepperwill not have anyservices. Althoughtracesdo collect APM data.
- However, the data schema named
Custom sourcewill display1-linghengqian-smoke-testsnormally. From beginning to end, the Spring Boot application did not throw any exceptions.
Expected behavior
A clear and concise description of what you expected to happen.
- The data schema named
Data Prepperin theobservability->traces->servicespage can display services normally.
Screenshots
If applicable, add screenshots to help explain your problem.
- Already mentioned above.
Environment (please complete the following information):
- OS: [e.g. Ubuntu 20.04 LTS]
Ubuntu WSL 22.04.5 LTS - Version [e.g. 22]
opensearchproject/data-prepper:2.10.1
Additional context
Add any other context about the problem here.
- An interesting point is that neither https://github.com/opensearch-project/data-prepper/tree/main/examples/jaeger-hotrod nor https://github.com/opensearch-project/data-prepper/pull/4972 have this problem. Am I supposed to understand that for a data schema named
Data Prepper, the service map is only available when there are multiple services? - Also, the data schema named
Data Prepperseems to be able to directly accept the grpc data sent by the otlp exporter, so why does the OpenSearch website require the additional deployment of data prepper?otel/opentelemetry-collector-contribcan actually directly convert the Opentelemetry SDK data intoopensearch documents. Another additional topic is, wouldn’t it be better if the opensearch dashboard directly integrates thelogs,metrics, andtracessent by the Opentelemetry SDK into a single web page? Why splitlogs,metrics, andtracesinto different web pages?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the issue with the linked data-prepper-logs-test repository and the observability-traces#/services page using the stated OpenJDK, Docker, and Data Prepper versions. Compare the Data Prepper schema with the Custom source schema and the referenced examples; done means the Data Prepper schema displays the collected service normally.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, java
- Domain
- observability-sre
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100