spring-cloud / spring-cloud/spring-cloud-gateway
Cache ManagedChannel and shared call context in JsonToGrpcGatewayFilterFactory (fixes leak, ~2.6× throughput)
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 4.9k
- Forks
- 3.5k
- Avg merge
- 20h 57m
- Merged PRs (30d)
- 8
Description
Is your feature request related to a problem? Please describe.
Under load, JsonToGrpcGatewayFilterFactory leaks gRPC ManagedChannel instances and repeats expensive initialization on every request.
A 400-request / 50-concurrency load test against a route that uses this filter produces 182 the errors below:
ERROR i.g.i.ManagedChannelOrphanWrapper > cleanQueue :
*~*~*~ Previous channel ManagedChannelImpl{logId=7, target=localhost:6565}
was not shutdown properly!!! ~*~*~*
java.lang.RuntimeException: ManagedChannel allocation site
There are two underlying problems in the current factory:
ManagedChannelis created per request and never shut down.
createChannelChannel(host, port)is called from insideGRPCResponseDecoratorfor every exchange.
- Heavy per-route initialization runs per request.
GRPCResponseDecoratoris constructed for every exchange, and its constructor parses the proto descriptor file, builds theFileDescriptorgraph, buildsMethodDescriptor+ marshallers, and stuffs — all of which only need to be built once per route.
Describe the solution you'd like
Two changes to JsonToGrpcGatewayFilterFactory:
-
Cache
ManagedChannelperhost:porton the factory. Move the cache to aConcurrentHashMap<String, ManagedChannel>field onJsonToGrpcGatewayFilterFactory. -
Extract a
GrpcCallContextthat holds the parsed proto descriptors, theMethodDescriptor, the sharedJsonFormat.Parser/Printer, theObjectMapper, and the emptyObjectNode. Build it once inapply(Config)and reuse it for every request on that route.
Measured impact with the same hey load (400 req / 50 concurrency):
| Metric | Before | After | Change |
|---|---|---|---|
| Requests/sec | 688.66 | 1805.41 | +162% |
| Average latency | 62.4 ms | 23.3 ms | −63% |
| p50 | 47.7 ms | 18.7 ms | −61% |
| p95 | 145.7 ms | 57.6 ms | −60% |
| p99 | 175.9 ms | 86.1 ms | −51% |
| Channel-leak errors | 182 | 0 | — |
Describe alternatives you've considered
N/A
Additional context
- Related issue: #3120. PR #3122 was approved in April 2024 but has been unmerged for over a year, and as noted above does not actually resolve the leak.
- I'm happy to open a PR with the changes above if this direction is acceptable.
Environment
- Spring Boot 3.4.5
- Spring Cloud 2024.0.1 (Moorgate)
- Spring Cloud Gateway 4.2.x
- grpc-java 1.69.0
- Netty 4.1.130.Final
- Java 17
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in JsonToGrpcGatewayFilterFactory, tracing createChannelChannel(host, port), GRPCResponseDecorator, and apply(Config). Verify how per-request channels and descriptor setup are currently created, then run the reported load test or existing gateway tests. Done means channels are reused and safely managed, route context is built once, and the leak errors no longer appear.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- grpc, java, spring-boot
- Domain
- api, backend, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100