apache / apache/incubator-pegasus

replication data consistency verify tool

Open
#304 0 comments 0 reactions 0 assignees View on GitHub
component/duplication type/enhancement
Dominant language
C++
Stars
2.1k
Forks
328
PR merge metrics
No merged PRs in 30d

Description

功能:对建立了replication关系的两个表进行数据验证,保证数据的一致性

使用场景:对于建立了replication关系的两个表,可以定期(譬如一周)在低峰时段进行数据一致性验证

注意点:
* 要考虑异步复制的延迟,譬如可以考虑获取数据的timestamp,对最近1小时写入的数据不进行比较;
* 可以考虑分布式并发执行,每个并发只比较一个partition,提升速度;

Contributor guide

Open the contributing guide

Research direction

No source file, test, or entry point is named. Start by clarifying how replication relationships, partitions, timestamps, and asynchronous replication delay are represented, then define the comparison and distributed execution semantics. Done means a usable periodic tool can verify replicated table consistency while excluding data still within the replication-delay window.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
databases, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.