spring-cloud / spring-cloud/spring-cloud-gateway
Performance issue with micrometer and otel baggage
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 4.9k
- Forks
- 3.5k
- Avg merge
- 20h 57m
- Merged PRs (30d)
- 8
Description
Describe the bug
I'm attempting to add several OpenTelemetry baggage items to the current tracing context using Micrometer and OpenTelemetry in a spring-cloud-starter-gateway-server-webflux application, but this appears to be significantly impacting throughput. It degrades from ~108 RPS to ~51 RPS 😟
Before baggage:
After baggage:
I have followed the steps given in the micrometer documentation page, i.e. by using ReactorBaggage and ObservationAwareBaggageThreadLocalAccessor
Sample
id("org.springframework.boot") version "3.5.0"
id("io.spring.dependency-management") version "1.1.7"
extra["springCloudVersion"] = "2025.0.0"
implementation("io.micrometer:micrometer-tracing")
implementation("io.micrometer:micrometer-tracing-bridge-otel")
Registering ObservationAwareBaggageThreadLocalAccessor:
@Bean
fun otelWebFilter(
tracer: Tracer,
registry: ObservationRegistry,
): OtelWebFilter {
ContextRegistry.getInstance()
.registerThreadLocalAccessor(ObservationAwareBaggageThreadLocalAccessor(registry, tracer))
ContextRegistry.getInstance().registerThreadLocalAccessor(ObservationAwareSpanThreadLocalAccessor(tracer))
return OtelWebFilter(tracer)
}
ReactorBaggage with contextWrite:
class OtelWebFilter(private val tracer: Tracer) : WebFilter {
private val logger = LoggerFactory.getLogger(OtelWebFilter::class.java)
override fun filter(
exchange: ServerWebExchange,
chain: WebFilterChain
): Mono<Void> {
val baggage = mapOf(
"username" to exchange.request.headers.getFirst("X-Username").orEmpty(),
"tenantId" to exchange.request.headers.getFirst("X-Tenant-Id").orEmpty(),
)
return chain.filter(exchange)
.doOnSubscribe { logger.debug("Setting baggage for path: {}", exchange.request.path) }
.doOnSuccess { logger.debug("Setting baggage completed") }
.contextWrite(ReactorBaggage.append(baggage))
}
}
Please look into it and let me know if you need further details
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the report with the Kotlin Spring Cloud Gateway sample, starting at the OtelWebFilter, ReactorBaggage.contextWrite call, and registered ObservationAware baggage accessors. Compare throughput with and without baggage and context propagation; done means the performance cause is isolated and a validated fix or clear finding is reported.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, spring, spring-boot
- Domain
- api, backend, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100