List actions are susceptible to GRPC message limits
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 3.7k
- Forks
- 676
- Avg merge
- 4d 9h
- Merged PRs (30d)
- 6
Description
Right now all list operations can trigger max message sizes in GRPC.
Of course this message size can be increased, however this is a problem in that these operations do not scale well and requires lots of allocations in both the client and the server.
We should investigate other means of handling lists.
A couple of alternatives. maybe someone else has another idea:
- Pagination
- Streaming responses
I think pagination is difficult to do reliably, particularly in a system with constant change and requires storing some extra state in both the client and the server.
A streaming response would require new API's but does not require juggling state as pagination would.
Containerd recently addressed a similar concern and decided to add streaming API's for lists here: https://github.com/containerd/containerd/pull/2347
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue names no files, tests, or entry points. Start by inventorying the existing list APIs and their gRPC message-size behavior, then review the linked containerd streaming API change. Done requires a decided approach and scoped API and implementation work, but this issue does not define that scope.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, grpc
- Domain
- api, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100