apache / apache/kvrocks

Disable the slot compaction while the cluster slot is migrating

Open
#2,079 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
C++
Stars
4.4k
Forks
658
Avg merge
1d 20h
Merged PRs (30d)
10

Description

### Search before asking

- [X] I had searched in the [issues](https://github.com/apache/kvrocks/issues) and found no similar issues.

### Motivation

Currently, we allow to run compaction even though the slot is migrating,
so it may cause data inconsistent in the corner case like below:

1. The cluster migration was started at T1 with the snapshot
2. The key 'foo' expired at T2, but it's extended between T1 and T2 to T3
3. Then the slot starts migrating the key 'foo' after T2, then it will be skipped since it's expired
4. Finally, the extended time write batch will be applied by iterating WAL items, but it will miss the subkeys since all of them are skipped.

This issue was found by @caipengbo, just record to prevent forgetting.

### Solution

Don't compact the slot if it's migrating, and also migrate them if it's expired.

### Are you willing to submit a PR?

- [ ] I'm willing to submit a PR!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by tracing the slot compaction and slot-migration entry points, including the WAL iteration path described in the issue. Verify the interaction between expiration, migration, and compaction, then confirm that migrating slots are not compacted and that expired keys are still migrated without losing subkeys.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
databases, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.