grpc / grpc/grpc-dotnet

Should GrpcClient use a segmented buffer for large messages?

Open
#2,665 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
C#
Stars
4.5k
Forks
836
Avg merge
6d 3h
Merged PRs (30d)
7

Description

### Is your feature request related to a problem? Please describe.

- Grpc clients read messages into a buffer rented from ArrayPool.Shared, see [here](https://github.com/grpc/grpc-dotnet/blob/master/src/Grpc.Net.Client/Internal/StreamExtensions.cs#L113)
- ArrayPool.Shared only pools arrays of up to 1024*1024 elements. Meaning, any message larger than 1MB is stored in a newly allocated array that lives outside of the array pool
- To make matters worse, buffers in those size regions immediately end up on the LOH
- That can cause a lot of LOH pollution for clients that deal with large messages

### Describe the solution you'd like

- Instead of trying to load everything into one big buffer, the client should possibly try to read these into chunks, each chunk no larger than 1mb and rented from the array pool
(- Potentially, it'd also be similar to leverage Microsoft.IO.RecylableMemoryStream here, though I imagine you may not want to add a dependency)
- The buffered result is already now passed down as a ReadOnlySequence of bytes already, which should work well here

### Describe alternatives you've considered
- Services can somewhat circumvent this problem by allowing to deliver large data spread across several messages
- This solution is not very ideal though as

1. Every endpoint/message needs explicit support for it
2. It doesn't work for unary calls
3. Payload size is oftentimes not easy to predict, so it's not trivial for client/server to choose a sensible model that guarantees messages are close to, but just below 1mb

Contributor guide

Open the contributing guide

Research direction

Start with src/Grpc.Net.Client/Internal/StreamExtensions.cs at the linked buffer-reading code, then trace how the buffered result is passed as a ReadOnlySequence. Determine the design and validation needed for pooled chunks no larger than 1 MB, with large messages avoiding a single large allocation; the issue does not name specific tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
api
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.