Correct the implemention of drainer mysql checkpoint
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 291
- Forks
- 131
- Avg merge
- 5m
- Merged PRs (30d)
- 2
Description
Feature Request
Is your feature request related to a problem? Please describe:
There are some problems with implemention of drainer mysql checkpoint, one mysql checkpoint example:
mysql> select * from tidb_binlog.checkpoint;
+---------------------+---------------------------------------------------------------------------------------------------------+
| clusterID | checkPoint |
+---------------------+---------------------------------------------------------------------------------------------------------+
| 6782769820199954307 | {"commitTS":413989456971300865,"ts-map":{"master-ts":413989358457585669,"slave-ts":413989440066945029}} |
| 6782784420681659503 | {"commitTS":414399178233348097,"ts-map":{"master-ts":414398077844324967,"slave-ts":414398082560032770}} |
+---------------------+---------------------------------------------------------------------------------------------------------+
2 rows in set (0.00 sec
there're two columns in checkpoint table
clusterIDwhich is primary keycheckpointwhich contain the specific checkpoint value
, andclusterIDis fetched from PD service of theupstream tidb cluster
the implementation brings three problems:
- how to fetch
clusterIDwhile the upstream cluster is completely down - if user want to deplay multiple drainer to replicate different tables from one upstream tidb cluster to the same downstream, they must specify different checkpoint schema and table. It's unreasonable, and user can't always figure out the weird logic
Describe the feature you'd like:
maybe we let user to specify the node-id for every drainer, and store node-id into checkpoint
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue names no files or tests. Start by locating drainer's MySQL checkpoint handling and its PD clusterID lookup, then examine how upstream unavailability and multiple drainers sharing one downstream are handled. Done should be an agreed checkpoint identity and storage design that addresses both cases, with tests covering them.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, mysql
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100