dragonflydb / dragonflydb/dragonfly
Improve memory defragment
- Dominant language
- C++
- Stars
- 31.5k
- Forks
- 1.3k
- Avg merge
- 1d 10h
- Merged PRs (30d)
- 137
Description
## Issue
Sometimes it's needed to explicitly defragment a dragonfly node via `memory defragment` command. This flow is stateful and it has no guarantees that it will attempt to defragment all the items in the dash table. Instead, it caches the last cursor state and subsequent calls to `memory defragment` act as a "step" in the defragment traversal. The main issue here is an API one; the command `memory defragment` does not return back to the client what portion (the cursror and what iteration/attempt per db) of the dbtable was traversed. As a consequence the client can't know how many times it needs to call `memory defragment` to complete the traversal.
Side note, the defrag is also a `background task` so we can keep a unique_id per iteration. That way the client will know if the traversal advanced from the background.
Contributor guide
Research direction
Start at the `memory defragment` command and the background-task flow described in the issue. Trace how the cursor, iteration or attempt per database, and unique_id state are maintained; done means the client can observe traversal progress and determine when the full traversal is complete.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- backend-api-design, databases
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100