Redwood and ssd TRIM
- Dominant language
- C++
- Stars
- 16.7k
- Forks
- 1.6k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 126
Description
A Redwood instance is a single file, and it tracks and reuses free space internally - it does NOT return free space back to the filesystem at any time. Redwood does not store metadata necessary to relocate free space over time to the end of the file, which would be required to enable truncating the file to remove the free space. This was an intentional design choice because the I/O to maintain such metadata and perform free space consolidation is unnecessary in the context of a filesystem volume dedicated for use as an FDB storage engine.
Because space is not returned to the filesystem, normal SSD TRIM support by filesystems will not result in returning freed Redwood pages to the SSD for use in wear leveling.
To (hopefully) regain trimming of unused space, the plan is to use a combination of `fallocate()` of freed pages with the `FALLOC_FL_ZERO_RANGE` flag, which should mark the blocks that the space occupies as "unwritten" from the filesystem perspective, which should then cause `fstrim` (to be run periodically on the fs) to issue TRIM commands to the disk.
I can't find documentation that this will actually work, however, so it needs to be tested.
The solution above is commonly referred to as periodic discard, but it would be great if instead we could get continuous discard behavior via `FALLOC_FL_ZERO_RANGE` and the `discard` mount option, but as far as I can tell this is not yet the case.
Contributor guide
Research direction
Start by testing Redwood's freed-page behavior with Linux fallocate() using FALLOC_FL_ZERO_RANGE, then run fstrim on the filesystem and observe whether the corresponding SSD blocks are discarded. Compare this with the discard mount option and document whether periodic or continuous discard is feasible; done means the filesystem and device behavior is verified and the implementation path is clear.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, linux
- Domain
- databases, operating-systems, performance
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100