apache / apache/kvrocks

Flaky test due to the data mismatch in kvrocks2redis consistency check

Open
#3,153 1 comment 0 reactions 0 assignees View on GitHub
A-ci bug
Dominant language
C++
Stars
4.4k
Forks
658
Avg merge
1d 20h
Merged PRs (30d)
10

Description

### Search before asking

- [x] I had searched in the [issues](https://github.com/apache/kvrocks/issues) and found no similar issues.

### Version

OS: Unknown
Kvrocks version: unstable (commit 1745d503)

### Minimal reproduce step

None

### What did you expect to see?

Consistency check should pass and destination data should match source data for all keys.

### What did you see instead?

Traceback (most recent call last):
File "/home/runner/work/kvrocks/kvrocks/utils/kvrocks2redis/tests/check_consistency.py", line 137, in
redis_comparator.compare_redis_data(args.key_file)
File "/home/runner/work/kvrocks/kvrocks/utils/kvrocks2redis/tests/check_consistency.py", line 91, in compare_redis_data
self._import_and_compare(100)
File "/home/runner/work/kvrocks/kvrocks/utils/kvrocks2redis/tests/check_consistency.py", line 122, in _import_and_compare
raise AssertionError(f"Data mismatch for key '{key}': source data: '{src_data}' destination data: '{dst_data}'")
AssertionError: Data mismatch for key 'key_49': source data: 'value_49' destination data: 'None'

### Anything Else?

WARNING: No config file specified, default configuration applied.
In order to specify a config file, use `kvrocks -c /path/to/kvrocks.conf`.
Test data-dir: /home/runner/work/kvrocks/kvrocks/kvrocks2redis-ci-data
daemonize: yes
output-dir: ./
namespace.__namespace: 127.0.0.1 6379
All case passed (except for data mismatch).

### Are you willing to submit a PR?

- [ ] I'm willing to submit a PR!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with utils/kvrocks2redis/tests/check_consistency.py, especially compare_redis_data and _import_and_compare, then run the kvrocks2redis consistency check using the reported test data directory. Trace why key_49 has source data but destination data is None. Done means the consistency check passes without data mismatches and all cases pass.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, redis
Domain
databases, testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.