grpc / grpc/grpc-java

Handle run-away netty buffer thread-local caches

Open
#4,317 8 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
12.1k
Forks
4k
Avg merge
2d 17h
Merged PRs (30d)
37

Description

gRPC uses ByteBufs from many non-Netty threads for message serialization. When the application has many threads, the per-thread caches can get out of hand.

Start of relevant discussion: https://gitter.im/grpc/grpc?at=5ac698877c3a01610db0f27b
User solution: https://gitter.im/grpc/grpc?at=5ac796016d7e07082bc71e0a
It references: netty/netty#5930

#1758 is related, as it seems allocating the per-thread cache has a noticeable cost. I thought there may be another related issue, but couldn't find it; maybe I'm thinking of that Netty one.

@jpitz had success just outright disabling caching for non-Netty threads. I'm slightly surprised that didn't cause a CPU increase, as that would seem to mean our direct buffers are no longer cached. If our benchmarks show similar behavior, then SGTM.

However, I also wonder if we should swap to heap buffers for message serialization and framing. Especially for small messages. That might hide most of the cost of removing the cache, since for larger messages the cost of direct buffer allocation is likely dwarfed by any copying.

CC @carl-mastrangelo

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.