spring-cloud / spring-cloud/spring-cloud-gateway

Cache ManagedChannel and shared call context in JsonToGrpcGatewayFilterFactory (fixes leak, ~2.6× throughput)

Open
#4,164 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

help wanted waiting-for-triage
Dominant language
Java
Stars
4.9k
Forks
3.5k
Avg merge
20h 57m
Merged PRs (30d)
8

Description

Is your feature request related to a problem? Please describe.

Under load, JsonToGrpcGatewayFilterFactory leaks gRPC ManagedChannel instances and repeats expensive initialization on every request.

A 400-request / 50-concurrency load test against a route that uses this filter produces 182 the errors below:

ERROR i.g.i.ManagedChannelOrphanWrapper > cleanQueue :
  *~*~*~ Previous channel ManagedChannelImpl{logId=7, target=localhost:6565}
  was not shutdown properly!!! ~*~*~*
java.lang.RuntimeException: ManagedChannel allocation site

There are two underlying problems in the current factory:

  1. ManagedChannel is created per request and never shut down.
  • createChannelChannel(host, port) is called from inside GRPCResponseDecorator for every exchange.
  1. Heavy per-route initialization runs per request.
  • GRPCResponseDecorator is constructed for every exchange, and its constructor parses the proto descriptor file, builds the FileDescriptor graph, builds MethodDescriptor + marshallers, and stuffs — all of which only need to be built once per route.

Describe the solution you'd like

Two changes to JsonToGrpcGatewayFilterFactory:

  1. Cache ManagedChannel per host:port on the factory. Move the cache to a ConcurrentHashMap<String, ManagedChannel> field on JsonToGrpcGatewayFilterFactory.

  2. Extract a GrpcCallContext that holds the parsed proto descriptors, the MethodDescriptor, the shared JsonFormat.Parser / Printer, the ObjectMapper, and the empty ObjectNode. Build it once in apply(Config) and reuse it for every request on that route.

Measured impact with the same hey load (400 req / 50 concurrency):

Metric Before After Change
Requests/sec 688.66 1805.41 +162%
Average latency 62.4 ms 23.3 ms −63%
p50 47.7 ms 18.7 ms −61%
p95 145.7 ms 57.6 ms −60%
p99 175.9 ms 86.1 ms −51%
Channel-leak errors 182 0

Describe alternatives you've considered

N/A

Additional context

  • Related issue: #3120. PR #3122 was approved in April 2024 but has been unmerged for over a year, and as noted above does not actually resolve the leak.
  • I'm happy to open a PR with the changes above if this direction is acceptable.

Environment

  • Spring Boot 3.4.5
  • Spring Cloud 2024.0.1 (Moorgate)
  • Spring Cloud Gateway 4.2.x
  • grpc-java 1.69.0
  • Netty 4.1.130.Final
  • Java 17

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in JsonToGrpcGatewayFilterFactory, tracing createChannelChannel(host, port), GRPCResponseDecorator, and apply(Config). Verify how per-request channels and descriptor setup are currently created, then run the reported load test or existing gateway tests. Done means channels are reused and safely managed, route context is built once, and the leak errors no longer appear.

Written by the indexing model from the issue text.

Assessment

Tech stack
grpc, java, spring-boot
Domain
api, backend, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.