debezium / debezium/dbz

Use the timestamp when LCR event happen in database to assign the timestamp of the SourceRecord received [DBZ-1585]

Open
#265 0 comments 0 reactions 0 assignees View on GitHub
component/core-library migrated-from-jira type/enhancement
Dominant language
HTML
Stars
6
Forks
8
Avg merge
2d 19h
Merged PRs (30d)
1

Description

Migrated from [DBZ-1585](https://issues.redhat.com/browse/DBZ-1585)

hi , in my case, I use the mysql connector and *ExtractNewRecordState* transform to extract the LCR and use JDBC connector to load the event message to other mysql server.

Now I want to do some end-to-end latency estimation on JDBC sink side. I find the timestamp of the SinkRecord received from Kafka topic is the the current timestamp when the source connector proccess the event and not the exact timestamp when the LCR happened in source database, in other words, I need the timestamp the mysql *bin-log* recorded.

So I expect the ExtractNewRecordState can provider some way I can replace the default timestamp with more exact event timestamp alternatively.

Contributor guide

Open the contributing guide

Research direction

The issue names the MySQL connector, MySQL binlog, ExtractNewRecordState, Kafka topic, and JDBC sink; start by tracing where the source record timestamp is assigned and how the transform handles it. Done means there is a testable way to preserve the binlog event timestamp for sink-side latency measurement.

Written by the indexing model from the issue text.

Assessment

Tech stack
kafka, mysql
Domain
data-engineering, databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.