Finish the missing lifecycle pieces in SingleFileIndexDirectory
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 189
Description
### Motivation
`pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/store/SingleFileIndexDirectory.java` still lists missing features near the class header:
- `newBuffer` opens a separate mapped buffer
- remove-index support is incomplete
- there is no way to abort or discard writes
The class already has mark-and-sweep comments for V3 cleanup, so the remaining lifecycle gaps are documented but unfinished.
### Scope
- Decide which missing behaviors should be supported directly versus explicitly unsupported.
- Implement or document remove-index and aborted-write handling.
- Add tests around partial writes and cleanup semantics.
- Update the class-level comments to match reality.
### Notes
Observed on `upstream/master` on April 15, 2026.
Contributor guide
Research direction
Start with pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/store/SingleFileIndexDirectory.java, especially the class-header notes and V3 mark-and-sweep comments. Trace the newBuffer, remove-index, and write paths, then define which lifecycle behaviors are supported or explicitly unsupported. Done means the implementation or documentation matches those decisions and tests cover partial writes and cleanup semantics.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, databases
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100