apache / apache/paimon

[Feature] Flink cdc produces tags based on the generation time of mysql binlog

Open
#4,080 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Java
Stars
3.4k
Forks
1.4k
Avg merge
1d 11h
Merged PRs (30d)
396

Description

### Search before asking

- [X] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.

### Motivation

In our company,user want produce daily tag base on the generation time of mysql binlog.Because it is more accurate to use the binlog generation time as the watermark.In mysql cdc binlog,I find a field `source.ts_ms` can be used for the generation time of mysql binlog .

### Solution

_No response_

### Anything else?

_No response_

### Are you willing to submit a PR?

- [ ] I'm willing to submit a PR!

Contributor guide

No contributing guide indexed for this repository

Research direction

No implementation files or tests are named. Start by locating the Flink CDC MySQL binlog handling and tracing the available source.ts_ms field; determine where the watermark and daily tags are produced. Done means tags use binlog generation time as the watermark, with coverage for the resulting behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, mysql
Domain
data-engineering, databases, stream-processing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.