spring-cloud / spring-cloud/spring-cloud-gateway
Expose `disposeInactivePoolsInBackground` configuration for HttpClient ConnectionProvider
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 4.9k
- Forks
- 3.5k
- Avg merge
- 20h 57m
- Merged PRs (30d)
- 8
Description
Problem Description
Spring Cloud Gateway (SCG) exposes HttpClientProperties to configure the underlying Reactor Netty connection pool, but it does not expose the disposeInactivePoolsInBackground(interval, inactivity) parameter from ConnectionProvider.Builder.
This causes a critical memory leak in environments with dynamic downstream instances (e.g., Kubernetes), where Pod IPs change frequently due to rolling deployments or auto-scaling.
Root Cause Analysis
Reactor Netty's PooledConnectionProvider maintains a ConcurrentHashMap<PoolKey, InstrumentedPool> named channelPools. A PoolKey is composed of:
remoteAddress(downstream IP:Port)pipelineKey(channelHash, derived fromHttpClientConfig)
Every time a downstream Pod is restarted or replaced, a new IP is assigned → a new PoolKey is created → a new SimpleDequePool is added to channelPools.
The critical problem: channelPools is never cleaned up automatically. The existing configuration options only clean up connections within a pool, not the PoolKey entries (pools themselves):
| Config property | Effect |
|---|---|
httpclient.pool.max-idle-time |
Evicts idle connections inside a pool ✅ |
httpclient.pool.max-life-time |
Evicts long-lived connections inside a pool ✅ |
httpclient.pool.eviction-interval |
Triggers periodic connection eviction ✅ |
disposeInactivePoolsInBackground |
Removes PoolKey entries for inactive pools ❌ Not exposed |
The disposeInactivePoolsInBackground method exists in PooledConnectionProvider (reactor-netty v1.1.x):
final void disposeInactivePoolsInBackground() {
toDispose = channelPools.entrySet()
.stream()
.filter(p -> p.getValue().metrics().isInactiveForMoreThan(poolInactivity))
.collect(Collectors.toList());
toDispose.forEach(e -> {
if (channelPools.remove(e.getKey(), e.getValue())) { // Actually removes the PoolKey!
e.getValue().dispose();
}
});
scheduleInactivePoolsDisposal();
}
This is triggered only when inactivePoolDisposeInterval is non-zero, which is set via ConnectionProvider.Builder.disposeInactivePoolsInBackground(interval, inactivity). SCG never sets this value, so it defaults to Duration.ZERO and the cleanup never runs.
Observed Impact
In our Kubernetes-based Spring Cloud Gateway deployment, we observed:
- 541,106 instances of
reactor.netty.internal.shaded.reactor.pool.SimpleDequePool - Occupying ~14.3 GB (94.34% of heap)
- GC Root path:
HttpResources → DefaultPooledConnectionProvider → channelPools (ConcurrentHashMap) → SimpleDequePool - Each
PoolKey.fqdncorresponds to a distinct downstream Pod IP that no longer exists
Heap dump analysis confirmed channelPools.size() equals the number of accumulated SimpleDequePool instances, directly corresponding to the number of distinct downstream IPs seen over the application's lifetime.
Current Workaround
Users must bypass HttpClientProperties entirely and register a custom ConnectionProvider bean:
@Bean
@Primary
public ConnectionProvider gatewayConnectionProvider() {
return ConnectionProvider.builder("gateway")
.maxConnections(500)
.maxIdleTime(Duration.ofSeconds(4))
.maxLifeTime(Duration.ofSeconds(60))
.evictInBackground(Duration.ofSeconds(10))
// Not available via application.properties!
.disposeInactivePoolsInBackground(
Duration.ofMinutes(5), // scan interval
Duration.ofMinutes(5) // inactivity threshold
)
.build();
}
This workaround works but requires users to know the internal reactor-netty API and loses the convenience of application.properties configuration.
Feature Request
Please expose disposeInactivePoolsInBackground in HttpClientProperties (or GatewayProperties) to allow configuration via:
spring.cloud.gateway.httpclient.pool.inactive-pool-dispose-interval=5m
spring.cloud.gateway.httpclient.pool.pool-inactivity=5m
And wire it into HttpClientAutoConfiguration when building the ConnectionProvider:
// In HttpClientAutoConfiguration or similar
if (pool.getInactivePoolDisposeInterval() != null) {
builder.disposeInactivePoolsInBackground(
pool.getInactivePoolDisposeInterval(),
pool.getPoolInactivity()
);
}
Environment
- Spring Cloud Gateway:
3.x(based on Spring Boot 3 / SCG 4.x series) - Reactor Netty:
1.1.5 - Deployment: Kubernetes with frequent Pod rolling updates
References
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Read HttpClientProperties and HttpClientAutoConfiguration to trace how pool settings reach the ConnectionProvider.Builder. Check the existing configuration-binding and auto-configuration tests, then add the two inactive-pool settings and verify that configured intervals are passed to the builder and exposed through application properties.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, spring-boot
- Domain
- api, backend, performance
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100