cockroachdb / cockroachdb/cockroach
rpc: experiment with different codecs (compression algorithms)
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
We currently use snappy compression for our gRPC RPCs. This decision was made years ago, and it might be worth revisiting it with the goal to see if it could be beneficial to switch to another codec. It's likely that snappy is a good choice, but worth a look anyway. Archaeology:
- https://github.com/cockroachdb/cockroach/pull/638
- https://github.com/cockroachdb/cockroach/pull/14239
Uber had an interesting post[^1] for a model serving workload. While likely not representative of our workloads (it looks like they were serving large blobs), it's a good introduction to the topic.
I think in principle we can also switch codecs based on the payload size (and expected response size) but that's likely overkill.
[^1]: https://careersatdoordash.com/blog/enabling-efficient-machine-learning-model-serving/
Jira issue: CRDB-45130
Contributor guide
Research direction
Start by reading the two linked historical pull requests and the referenced workload discussion to understand the existing snappy choice. Locate the gRPC compression path and establish representative benchmarks for alternative codecs; the work is complete when measured results support retaining or changing the codec choice.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, grpc
- Domain
- backend-api-design, distributed-systems, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100