elastic / elastic/apm-agent-java

Spring Webflux performance issues in 1.57

Open
#4,573 4 comments 0 reactions 1 assignee Claimed by @SylvainJuge View on GitHub
agent-java community triage
Dominant language
Java
Stars
594
Forks
338
Avg merge
1d 13h
Merged PRs (30d)
25

Description

## Describe the bug
The new `co.elastic.apm.agent.reactor.SubscriptionCancelInstrumentation` has significant impact on performance in Spring Webflux project, especially when using alongside something that runs with low latency like Redis causing exceptions like this:

```
Unable to acquire connection! java.util.concurrent.CompletableFuture@18e0306a[Completed exceptionally: java.util.concurrent.CancellationException]Increase connection pool size. Node source: NodeSource [slot=0, addr=null, redisClient=null, redirect=null, entry=null], command: (EXPIRE), params: [[115, 112, 114, 105, 110, 103, 58, 115, 101, 115, ...], 1209900] after 1 of 2 retry attempts
at org.redisson.command.RedisExecutor.lambda$scheduleRetryTimeout$0(RedisExecutor.java:309)
at io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:717)
at io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34)
at io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:705)
at io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:791)
at io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:501)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:1474)
```

I tested the throughput of the ["non-instrumentation"](https://github.com/elastic/apm-agent-java/pull/4556/changes/1b67ee2ed22a81c72fb43037d34b9dee993274b3) solution that we discussed [here](https://github.com/elastic/apm-agent-java/pull/4556#discussion_r3919133037) and I saw ~10x improvement in throughput or latency of http calls of the service

## Steps to reproduce
Steps to reproduce the behavior:
1. Create Spring Boot 4 Webflux project with some simple API and use cache using Redis in it
2. Compare throughput of the service http calls with and without APM client version 1.57

## Expected behavior
Throughput should be very similar ( not 10x diff )

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.