Compaction reliability
- Dominant language
- Java
- Stars
- 107
- Forks
- 29
- Avg merge
- 19h 46m
- Merged PRs (30d)
- 141
Description
### User Story
As a user of Sleeper, I want compaction jobs to run deterministically and reliably, so that I know my data will be processed and won't be lost.
### Description / Background
We'd like to ensure that:
- Data scheduled into a compaction job will always be written to a new file.
- The new file replaces the old file atomically, such that no data is lost or duplicated.
- If data cannot be processed, it should be transparent to the user why this was the case, and what they can do about it.
- If a compaction job cannot be processed for some temporary reason, the underlying data will still be correctly processed.
### Technical Notes / Implementation Details
See sub-issues. We may make notes here on potential issues that have not yet been raised.
Contributor guide
Assessment
This issue has not been assessed yet.