grpc / grpc/grpc-java

Address key protobuf performance issues

Open
#1,054 43 comments 6 reactions 1 assignee Claimed by @nmittler View on GitHub
P2
Dominant language
Java
Stars
12.1k
Forks
4k
Avg merge
2d 17h
Merged PRs (30d)
37

Description

This is something of a coverall issue and should be factored out into separate issues for each individual piece of work. Listed here are a number of areas for improving the interaction between protobuf and GRPC to improve overall performance and reduce memory pressure on the system.

In rough order of priority....
- Implement some basic protobuf benchmarks independent of GRPC (scrape the web or extract Googles) as a framework for experimenting with optimizations
- Provide the ability to have a 0-copy approach for writing out large binary sequences (byte buf, file region etc) to the transport without copying
- Provide the ability to read protobufs so that they can be written out again with 0-copy for large sequences. This includes writes to transport and to files
- Document a pattern for transferring a large binary sequence as chunks and utility functions to chunk & dechunk those to application code. File transfer is a common use-case here
- Consider supporting deserialized protobuf backed by pooled buffers (speculative). Would require the notion of a releasable protobuf

@nmittler @ejona86 - Feel free to pile on / cc folks

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.