Use the timestamp when LCR event happen in database to assign the timestamp of the SourceRecord received [DBZ-1585]
- Dominant language
- HTML
- Stars
- 6
- Forks
- 8
- Avg merge
- 2d 19h
- Merged PRs (30d)
- 1
Description
Migrated from [DBZ-1585](https://issues.redhat.com/browse/DBZ-1585)
hi , in my case, I use the mysql connector and *ExtractNewRecordState* transform to extract the LCR and use JDBC connector to load the event message to other mysql server.
Now I want to do some end-to-end latency estimation on JDBC sink side. I find the timestamp of the SinkRecord received from Kafka topic is the the current timestamp when the source connector proccess the event and not the exact timestamp when the LCR happened in source database, in other words, I need the timestamp the mysql *bin-log* recorded.
So I expect the ExtractNewRecordState can provider some way I can replace the default timestamp with more exact event timestamp alternatively.
Contributor guide
Research direction
The issue names the MySQL connector, MySQL binlog, ExtractNewRecordState, Kafka topic, and JDBC sink; start by tracing where the source record timestamp is assigned and how the transform handles it. Done means there is a testable way to preserve the binlog event timestamp for sink-side latency measurement.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- kafka, mysql
- Domain
- data-engineering, databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100