Add ability to record a log of AsyncFileCached page accesses for some time window
- Dominant language
- C++
- Stars
- 16.7k
- Forks
- 1.6k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 126
Description
In support of https://github.com/apple/foundationdb/issues/1518, it would be nice to have an ability to log storage engine page accesses from a live cluster for use in evaluating different caching policies.
One easy to use form of this would be for the fdbcli "profile" command to support this via a subcommand. Each process will would accumulate page numbers accessed for the specified amount of time into memory and then write the collected data to a set of keys AFTER the profile period is complete (so as to not affect the workload during the collection).
The data could be logged to chunked key/value pairs with keys like `/filename/`.
Contributor guide
Research direction
Start with the fdbcli "profile" command and the AsyncFileCached storage-engine page-access path. Determine how a live cluster could collect page numbers for a specified window and write chunked key/value records only after collection ends. Done means the profile interface supports this workflow without affecting the workload during the collection period.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- cli, databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100