cockroachdb / cockroachdb/pebble
add a compression analyzer facility
- Dominant language
- Go
- Stars
- 6k
- Forks
- 584
- Avg merge
- 16h 35m
- Merged PRs (30d)
- 5
Description
This issue tracks adding a facility for analyzing data in real clusters. The goal is to get a good comparison between various compression algorithms and levels and use it to inform our suggested defaults or to add new adaptive compression algorithms.
We have two ways of doing this:
- online: we can sample blocks as they are written to or read from disk. For each sampled blocks, we run all experiments and retain statistics. This approach has the advantage of allowing us to accurately estimate CPU usage differences between algorithms within a specific workload. The disadvantage is that we can only produce data on clusters with versions that include this facility.
- "offline": we can add a CLI tool that looks at all relevant files from a store and samples blocks separately from any running process. This is easier to implement and provides a quicker way to obtain data, as a newer binary can be used just for this tool.
Jira issue: PEBBLE-442
Epic CRDB-49140
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.