Handle run-away netty buffer thread-local caches
- Dominant language
- Java
- Stars
- 12.1k
- Forks
- 4k
- Avg merge
- 2d 17h
- Merged PRs (30d)
- 37
Description
gRPC uses ByteBufs from many non-Netty threads for message serialization. When the application has many threads, the per-thread caches can get out of hand.
Start of relevant discussion: https://gitter.im/grpc/grpc?at=5ac698877c3a01610db0f27b
User solution: https://gitter.im/grpc/grpc?at=5ac796016d7e07082bc71e0a
It references: netty/netty#5930
#1758 is related, as it seems allocating the per-thread cache has a noticeable cost. I thought there may be another related issue, but couldn't find it; maybe I'm thinking of that Netty one.
@jpitz had success just outright disabling caching for non-Netty threads. I'm slightly surprised that didn't cause a CPU increase, as that would seem to mean our direct buffers are no longer cached. If our benchmarks show similar behavior, then SGTM.
However, I also wonder if we should swap to heap buffers for message serialization and framing. Especially for small messages. That might hide most of the cost of removing the cache, since for larger messages the cost of direct buffer allocation is likely dwarfed by any copying.
CC @carl-mastrangelo
Contributor guide
Assessment
This issue has not been assessed yet.