debezium / debezium/dbz

On connection retry seems the GTID positions are not checked [DBZ-1737]

Open
#282 0 comments 0 reactions 0 assignees View on GitHub
component/mysql-connector migrated-from-jira type/bug
Dominant language
HTML
Stars
6
Forks
8
Avg merge
2d 19h
Merged PRs (30d)
1

Description

Migrated from [DBZ-1737](https://issues.redhat.com/browse/DBZ-1737)

We're seeing strange behaviour. Mysql connector that is running and successfully producing events on restart starts to fail with error, that binlogs are not available anymore.

The error is correct - some non active gtid positions were indeed purged. But it was during failover already long time ago. So it seems reconnecting hides the real problem until next restart.

After real restart the connector actually checks mysql server values and fails.
2020-01-24 07:26:46.347
org.apache.kafka.connect.errors.ConnectException: The connector is trying to read binlog starting at GTIDs
ac917707-0327-11ea-87a2-fa163e4b55f4:1-7166031,ed5b8fc3-daab-11e8-b102-fa163e4b55f4:1-325053312,ed6c7f2a-daab-11e8-b595-fa163e0684a1:1-38230159
, but this is no longer available on the server.

But just few hours ago it "succeeded" restoring connection.
2020-01-24 03:09:53.541
And then INFO: Trying to restore lost connection to 10.222.2.102:3306
2020-01-24 03:09:53.685 s2w22l
INFO: Connected to 10.222.2.102:3306 at ac917707-0327-11ea-87a2-fa163e4b55f4:1-7166031,ed5b8fc3-daab-11e8-b102-fa163e4b55f4:1-325053312,ed6c7f2a-daab-11e8-b595-fa163e0684a1:1-38186365 (sid:107400, cid:16595316)
+ succeed and produced all events.

This also indicates that the strict binlog match and failure is not always needed - on restore lost connection flow it actually works.

Contributor guide

Open the contributing guide

Research direction

Start by tracing the MySQL connector's lost-connection restore path and comparing it with the restart path described in the logs. Check where GTID positions are validated during reconnect, then add coverage for purged positions and confirm that reconnect behavior reports the same failure as a fresh restart.

Written by the indexing model from the issue text.

Assessment

Tech stack
mysql
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.