Txn compaction recovery can leave entry logs unreclaimed after restart
- Dominant language
- Java
- Stars
- 2k
- Forks
- 976
- Avg merge
- 6d 15h
- Merged PRs (30d)
- 7
Description
### Problem
Txn compaction recovery can create the recovered entry log with a decimal file name, while `DefaultEntryLogger` later resolves the same log id with the normal hex file name.
Entry log file names are hex-based. For log id `500`, the expected file name is `1f4.log`.
### Code path
Normal txn compaction keeps the hex file name when it builds `finalLogFile`:
https://github.com/apache/bookkeeper/blob/cbb336702aaa17608d811a52ca7b03559e29cbb0/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/DefaultEntryLogger.java#L1316-L1326
`makeAvailable()` creates the final entry log at `finalLogFile`:
https://github.com/apache/bookkeeper/blob/cbb336702aaa17608d811a52ca7b03559e29cbb0/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/DefaultEntryLogger.java#L1370-L1386
Restart recovery rebuilds `finalLogFile` from a `*.compacted` file:
https://github.com/apache/bookkeeper/blob/cbb336702aaa17608d811a52ca7b03559e29cbb0/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/DefaultEntryLogger.java#L1424-L1478
Later scan resolves a numeric log id back to the hex file name:
https://github.com/apache/bookkeeper/blob/cbb336702aaa17608d811a52ca7b03559e29cbb0/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/DefaultEntryLogger.java#L977-L1005
### Example
Assume restart recovery sees this pending compaction file:
`1f4.log.78.compacted`
- `1f4` is the destination log id, hex for decimal `500`.
- recovery parses `1f4` as `500`.
- recovery builds the final file name from the numeric id and targets `500.log`.
- `makeAvailable()` creates `500.log`.
- later scan for log id `500` looks for `1f4.log`.
The created file and the lookup file do not match.
### Impact
Txn compaction recovery may not scan and finalize the recovered log. The `.compacted` marker and old source entry log can remain on disk, so disk space for that compaction is not reclaimed.
Ledger entry contents are not rewritten or changed.
### Fix
Rebuild the recovered final `.log` file name with the same hex format used by normal entry logs.
The fix also covers the upgrade case where an older recovery attempt already left a stale decimal-named log file while the `.compacted` marker is still present.
Contributor guide
Assessment
This issue has not been assessed yet.