dragonflydb / dragonflydb/dragonfly

Improve memory defragment

Open
#6,115 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
C++
Stars
31.5k
Forks
1.3k
Avg merge
1d 10h
Merged PRs (30d)
137

Description

## Issue

Sometimes it's needed to explicitly defragment a dragonfly node via `memory defragment` command. This flow is stateful and it has no guarantees that it will attempt to defragment all the items in the dash table. Instead, it caches the last cursor state and subsequent calls to `memory defragment` act as a "step" in the defragment traversal. The main issue here is an API one; the command `memory defragment` does not return back to the client what portion (the cursror and what iteration/attempt per db) of the dbtable was traversed. As a consequence the client can't know how many times it needs to call `memory defragment` to complete the traversal.

Side note, the defrag is also a `background task` so we can keep a unique_id per iteration. That way the client will know if the traversal advanced from the background.

Contributor guide

Open the contributing guide

Research direction

Start at the `memory defragment` command and the background-task flow described in the issue. Trace how the cursor, iteration or attempt per database, and unique_id state are maintained; done means the client can observe traversal progress and determine when the full traversal is complete.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend-api-design, databases
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.