open-telemetry / open-telemetry/opentelemetry-java-instrumentation
Multiple trace Id's generated from kafka consumer
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 2.6k
- Forks
- 1.2k
- Avg merge
- 2d 18h
- Merged PRs (30d)
- 228
Description
Describe the bug
We have deployed three services Service A, Service B and Service C. Service A interacts with Service B via API call and drops a message to Kafka Topic which is consumed by Service C. Service C consumes the record and calls external API call and inserts data into database. We observe multiple trace Id's generated for database and external API calls from kafka consumer instead of multiple span Id's.
Services Interaction: https://github.com/sunvuz/OpenTelemetry-poc/blob/30cdb74b4ac53bbba08fbc510e5900b3dc6bdf12/Kafka%20Consumer%20-New%20Traceid%20issue.PNG
Zipkin: Image with multiple trace Id's from consumer https://github.com/sunvuz/OpenTelemetry-poc/blob/30cdb74b4ac53bbba08fbc510e5900b3dc6bdf12/Kafka%20Consumer%20-New%20Traceid%20issue%20-Zipkin.PNG
Steps to reproduce
Sample Service's mimicking above behavior has been updated at : https://github.com/sunvuz/OpenTelemetry-poc
What did you expect to see?
Span Id for database and external API call Interaction instead of Trace Id's
What did you see instead?
New Trace Id's generated for database and external API interaction.
What version are you using?
v1.0.0 same is the behavior with 0.x.x versions
Environment
JDK 8+
OS : Linux
Additional context
Add any other context about the problem here.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the linked sample service and its Kafka consumer flow, then compare the trace and span behavior shown in the linked Zipkin image. Reproduce the database and external API interactions with version 1.0.0 or a 0.x.x version, and determine whether those interactions should remain under the consumer's trace. Done means the reported behavior is explained and corrected with verification of the resulting trace relationships.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, kafka
- Domain
- distributed-systems, observability
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100