Investigate how to handle examining files as part of compaction planning
- Dominant language
- Java
- Stars
- 1.2k
- Forks
- 487
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 13
Description
As part of selecting files to compact for user compactions the compaction selector plugin offered examining [summaries](https://github.com/apache/accumulo/blob/06c55fbea47f7ac376c7c93b48b76529e0d4a474/core/src/main/java/org/apache/accumulo/core/client/admin/compaction/CompactionSelector.java#L60) and [sample data](https://github.com/apache/accumulo/blob/06c55fbea47f7ac376c7c93b48b76529e0d4a474/core/src/main/java/org/apache/accumulo/core/client/admin/compaction/CompactionSelector.java#L71) as part of this decision. This data supported use cases like the [TooManyDeletesSelector](https://github.com/apache/accumulo/blob/06c55fbea47f7ac376c7c93b48b76529e0d4a474/core/src/main/java/org/apache/accumulo/core/client/admin/compaction/TooManyDeletesSelector.java#L66) which could trigger a full tablet compaction when the ratio of deletes in a tablet exceeded a certain threshold.
When the examination ran in tablet servers this examination had much more CPU and memory available than what will be available when running the examination in the manager. In #3513 which moved user compaction from tablet server to the manager this functionality was not implemented because of concerns about resources.
Contributor guide
Research direction
Start with CompactionSelector.java and TooManyDeletesSelector.java, then review the user-compaction changes in #3513. Investigate how summaries and sample data could be examined during manager-side compaction planning under its resource constraints. Done means the project has a decided approach for handling this examination functionality.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100