open-telemetry / open-telemetry/opentelemetry-java-instrumentation
Bug: gRPC Context incorrectly propagated to async threads via OpenTelemetry Context (with reproduction code)
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 2.6k
- Forks
- 1.2k
- Avg merge
- 2d 18h
- Merged PRs (30d)
- 228
Description
When using OpenTelemetry's gRPC instrumentation library with Context.current().wrap(runnable) to propagate OpenTelemetry context to async threads, the gRPC context is incorrectly propagated along with it. This causes errors when making gRPC calls from async threads because the gRPC context from the original request thread is incorrectly activated in the async thread.
To Reproduce:
##reproduction code repository : https://github.com/153952402/grpc-otel-context-bug-reproduction
cd grpc-otel-context-bug-reproduction
mvn spring-boot:run
In another terminal:
grpcurl -plaintext \
-import-path ./src/main/resources/proto \
-proto downstream_service.proto \
-d '{"id":"test-123","data":"test data"}' \
127.0.0.1:9090 \
com.example.reproduction.downstream.grpc.DownstreamService/Handle
What did you expect to see?
- The gRPC Context should NOT be propagated to async threads - only OpenTelemetry tracing context should be propagated.
- When checking
io.grpc.Context.current()in async threads, it should returnContext.ROOT. - Async threads should be able to make gRPC calls independently without errors.
What did you see instead?
- The gRPC Context IS incorrectly propagated to async threads via OpenTelemetry Context.
- When checking
io.grpc.Context.current()in async threads, it returns a non-ROOT context (the context from the original request thread). - When making gRPC calls from async threads, errors occur because:
- The gRPC context from the original request is still active
- This can cause deadline/cancellation states to be incorrectly propagated
- The context state may be inconsistent for the new gRPC call
What version and what artifacts are you using?
Artifacts:
opentelemetry-apiversion 1.32.0opentelemetry-sdkversion 1.32.0opentelemetry-grpc-1.6(instrumentation library) version 2.1.0-alphagrpc-netty-shadedversion 1.60.1grpc-protobufversion 1.60.1grpc-stubversion 1.60.1spring-boot-starter-webversion 3.2.0grpc-server-spring-boot-starterversion 2.15.0.RELEASE
Version:
- Java: 21
- Spring Boot: 3.2.0
- OpenTelemetry: 1.32.0
- OpenTelemetry Instrumentation (gRPC 1.6): 2.1.0-alpha
- gRPC: 1.60.1
Thank you for your attention to this issue
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by running the reproduction repository linked in the issue with the provided Maven and grpcurl commands, then inspect the gRPC instrumentation and OpenTelemetry context-propagation paths in this repository. Done means async threads retain the OpenTelemetry tracing context without inheriting the original gRPC context, report Context.ROOT, and can make independent gRPC calls without errors.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- grpc, java
- Domain
- distributed-systems, observability-sre
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100