Implement read cache for disk queue
- Dominant language
- C++
- Stars
- 16.7k
- Forks
- 1.6k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 126
Description
With FDB 6.2, spill-by-reference is now the default. Currently, the disk queue is still uncached, as it was previously a read-once file. In the case of a storage server failure, there won't be a high degree of locality (spatial or temporal) between storage servers catching up, so adding a cache wouldn't be beneficial in this case.
However, for multi-region clusters, all log routers are forced to stay within 5s of each other as they peek their respective log router tags. This means that there will be a high degree of temporal locality of disk queue reads when a remote region is trying to catch up. Adding a (preferably read-only) cache onto the disk queue should help greatly reduce the read amplification of spill-by-reference when catching up from remote DCs (when number of log routers is larger than number of satellites).
Contributor guide
Research direction
Start by locating the disk queue and spill-by-reference paths described in the issue, then trace how remote-region catch-up reads queue data. Determine where a read-only cache would fit and validate that multi-region catch-up reuses disk-queue reads without changing queue semantics. The issue names no file or test, so finding the relevant implementation and existing coverage is part of the work.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- databases, distributed-systems
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100