apache / apache/bookkeeper

Prioritize compaction of entry logs with the lowest amount of remaining usable data

Open
#3,389 0 comments 1 reaction 0 assignees View on GitHub
type/feature
Dominant language
Java
Stars
2k
Forks
976
Avg merge
6d 15h
Merged PRs (30d)
7

Description

**FEATURE REQUEST**

1. Please describe the feature you are requesting.

Prioritize compaction to free up more space faster.

2. Indicate the importance of this issue to you (blocker, must-have, should-have, nice-to-have).

must-have

3. Provide any additional detail on your proposed use case for this feature.

Looking at GarbageCollectorThread:

doCompactEntryLogs() iterates over entry logs in whatever natural order they happen to be, picks the first with usage below thresholds and starts compacting.
Do major compaction it means we can start compaction with an entry log at 80% utilization instead of e.g. one with 10%.
This can be easily fixed by building a PriorityQueue of entry logs, ordering by lowest utilization (meta.getUsage()) to free up more space sooner.
Building of the queue should not take too much time and can be combined with doGcEntryLogs() which iterates over all entries in entryLogMetaMap anyway; memory-wise it should be fine too.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.