influxdata / influxdata/influxdb
Implement Bonobo algorithm for influxdb, enabling 5 times better data compression
- Dominant language
- Rust
- Stars
- 31.7k
- Forks
- 3.7k
- Avg merge
- 13h 37m
- Merged PRs (30d)
- 8
Description
__Proposal:__
Improve the compression algorithm like the Bonobo timeseries data compression algorithm sugggest here https://github.com/JsBergbau/Bonobo-Timeseries-Data-Compression
__Current behavior:__
Due to the nature of floating point numbers in binary system and the data from the decimal system there is often a lot of change in data, requiring unnecessary disk space. Measurements showed that compression can be improved by a factor of 5.
__Desired behavior:__
Bonobo timeseries data compression as explained in https://github.com/JsBergbau/Bonobo-Timeseries-Data-Compression is implemented.
__Alternatives considered:__
Currently I don't know and better solution to gain so much compression improvement.
__Use case:__
Why is this important (helps with prioritizing requests)?
Saving data by the factor of 5 is a lot of improvement, especially for 32 bit systems where database size is quite limited, see https://github.com/influxdata/influxdb/pull/12362
Contributor guide
Research direction
Start by reading the linked Bonobo Timeseries Data Compression project and the referenced InfluxDB pull request. Identify the relevant InfluxDB storage and compression entry points, then determine how the proposed algorithm could be integrated and benchmarked. Done means the algorithm is implemented and the claimed compression improvement is measured.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100