temporalio / temporalio/temporal
Compress history_node->data cell
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 23.2k
- Forks
- 1.9k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 228
Description
Some data cells can be 2M or so large, which isn't great, considering this is just proto3 data, written once, rarely read. They should be compressed by default, perhaps with some minimum size that is worth the effort.
Example from a production workload:
It seems I can use this:
The PayloadCodec interface:
type PayloadCodec interface {
Encode(payloads []*commonpb.Payload) ([]*commonpb.Payload, error)
Decode(payloads []*commonpb.Payload) ([]*commonpb.Payload, error)
}
If approved, I'll give it a try.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the PayloadCodec interface and the history_node data-cell write and read paths. Determine where a minimum compression threshold belongs and verify that default compression preserves decoding behavior; done means large cells are compressed while smaller cells avoid unnecessary work.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- backend, distributed-systems
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100