Should GrpcClient use a segmented buffer for large messages?
- Dominant language
- C#
- Stars
- 4.5k
- Forks
- 836
- Avg merge
- 6d 3h
- Merged PRs (30d)
- 7
Description
### Is your feature request related to a problem? Please describe.
- Grpc clients read messages into a buffer rented from ArrayPool.Shared, see [here](https://github.com/grpc/grpc-dotnet/blob/master/src/Grpc.Net.Client/Internal/StreamExtensions.cs#L113)
- ArrayPool.Shared only pools arrays of up to 1024*1024 elements. Meaning, any message larger than 1MB is stored in a newly allocated array that lives outside of the array pool
- To make matters worse, buffers in those size regions immediately end up on the LOH
- That can cause a lot of LOH pollution for clients that deal with large messages
### Describe the solution you'd like
- Instead of trying to load everything into one big buffer, the client should possibly try to read these into chunks, each chunk no larger than 1mb and rented from the array pool
(- Potentially, it'd also be similar to leverage Microsoft.IO.RecylableMemoryStream here, though I imagine you may not want to add a dependency)
- The buffered result is already now passed down as a ReadOnlySequence of bytes already, which should work well here
### Describe alternatives you've considered
- Services can somewhat circumvent this problem by allowing to deliver large data spread across several messages
- This solution is not very ideal though as
1. Every endpoint/message needs explicit support for it
2. It doesn't work for unary calls
3. Payload size is oftentimes not easy to predict, so it's not trivial for client/server to choose a sensible model that guarantees messages are close to, but just below 1mb
Contributor guide
Research direction
Start with src/Grpc.Net.Client/Internal/StreamExtensions.cs at the linked buffer-reading code, then trace how the buffered result is passed as a ReadOnlySequence. Determine the design and validation needed for pooled chunks no larger than 1 MB, with large messages avoiding a single large allocation; the issue does not name specific tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- api
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100