debezium / debezium/dbz

Error during binlog processing [DBZ-9463]

Open
#89 0 comments 0 reactions 0 assignees View on GitHub
component/mariadb-connector migrated-from-jira type/bug
Dominant language
HTML
Stars
6
Forks
8
Avg merge
2d 19h
Merged PRs (30d)
1

Description

Migrated from [DBZ-9463](https://issues.redhat.com/browse/DBZ-9463)

h1. Bug report

Using debezium source connector to *mariadb* for shiping binlogs to *nats-jetstream*
everything goes fine until I get 
{code:java}
io.debezium.connector.binlog.BinlogStreamingChangeEventSource","level":"WARN","message":"Error during binlog processing.{code}
 
h2. What Debezium connector do you use and what version?

debezium.source.connector.class=io.debezium.connector.mariadb.MariaDbConnector
quay.io/debezium/server:3.2.1
h2. What is the connector configuration?

 
{code:java}
#Debezium Source: MariaDB
debezium.source.connector.class=io.debezium.connector.mariadb.MariaDbConnector
debezium.source.database.hostname=hostname.dk5.tv
debezium.source.database.port=3306
debezium.source.database.user=debezium_slip_log
debezium.source.database.password=PASS
debezium.source.database.server.id=1337
debezium.source.database.include.list=advbet_web_test
debezium.source.table.include.list=advbet_web_test.slip_log
#Debezium Topic/Format
debezium.source.topic.prefix=debezium
debezium.source.topic.capture.mode=change_streams
debezium.format.value=json
#Offset and Schema History Storage
debezium.source.offset.storage.file.filename=/debezium/data/offsets.dat
debezium.source.schema.history.internal=io.debezium.storage.file.history.FileSchemaHistory
debezium.source.schema.history.internal.file.filename=/debezium/data/schema-history.dat
debezium.source.include.schema.changes=true
debezium.source.snapshot.mode=initial
#Error Handling
debezium.source.skip.messages.without.change=true
debezium.source.event.processing.failure.handling.mode=warn
debezium.source.event.converting.failure.handling.mode=warn
debezium.source.skipped.operations=u,d,t
#Logging
debezium.log.level=TRACE
#Debezium Sink: NATS JetStream
debezium.sink.type=nats-jetstream
debezium.sink.nats-jetstream.url=tls://hostname01.dk5.tv:4222,tls://hostname02.dk5.tv:4222,tls://hostname03.dk5.tv:4222
#TLS Authentication
debezium.sink.nats-jetstream.auth.tls.keystore=/debezium/config/keystore.p12
debezium.sink.nats-jetstream.auth.tls.keystore.password=PASS
debezium.sink.nats-jetstream.auth.tls.password=PASS
#JetStream Stream Settings
debezium.sink.nats-jetstream.create-stream=false
debezium.sink.nats-jetstream.stream=db_changes_stream
debezium.sink.nats-jetstream.subjects=debezium,debezium.>
debezium.sink.nats-jetstream.auth.user=debezium
debezium.sink.nats-jetstream.auth.password=PASS
{code}
h2. What is the captured database version and mode of deployment?

on hetzner cloud hardware using  kvm
MariaDB-server.x86_64               10.11.14-1.el9
h2. What behavior do you expect?

h2. What behavior do you see?

Using debezium source connector to *mariadb* for shiping binlogs to *nats-jetstream*
everything goes fine until I get 

 
{code:java}
{"timestamp":"2025-09-12T13:15:23.511281715Z","sequence":16415,"loggerClassName":"org.slf4j.impl.Slf4jLogger","loggerName":"io.debezium.connector.binlog.BinlogStreamingChangeEventSource","level":"WARN","message":"A deserialization failure event arrived","threadName":"blc-adv-prod-online-test-srv01.duok5.tv:3306","threadId":46,"mdc":{"dbz.connectorLogicalName":"debezium","dbz.taskId":"0","dbz.connectorType":"MariaDB","dbz.connectorContext":"binlog"},"ndc":"","hostName":"64a025d9b9c0","processName":"/usr/lib/jvm/java-21-openjdk-21.0.8.0.9-1.el8.x86_64/bin/java","processId":1,"exception":{"refId":1,"exceptionType":"com.github.shyiko.mysql.binlog.event.deserialization.EventDataDeserializationException","message":"Failed to deserialize data of EventHeaderV4{timestamp=1757682923000, eventType=WRITE_ROWS, serverId=2420921427, headerLength=19, dataLength=2005, nextPosition=1147945, flags=0}","frames":[{"class":"com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer","method":"deserializeEventData","line":343},{"class":"com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer","method":"nextEvent","line":246},{"class":"io.debezium.connector.binlog.BinlogStreamingChangeEventSource$1","method":"nextEvent","line":425},{"class":"com.github.shyiko.mysql.binlog.BinaryLogClient","method":"listenForEventPackets","line":1082},{"class":"com.github.shyiko.mysql.binlog.BinaryLogClient","method":"connect","line":657},{"class":"com.github.shyiko.mysql.binlog.BinaryLogClient$7","method":"run","line":959},{"class":"java.lang.Thread","method":"run","line":1583}],"causedBy":{"exception":{"refId":2,"exceptionType":"com.github.shyiko.mysql.binlog.event.deserialization.MissingTableMapEventException","message":"No TableMapEventData has been found for table id:6276. Usually that means that you have started reading binary log 'within the logical event group' (e.g. from WRITE_ROWS and not proceeding TABLE_MAP","frames":[{"class":"com.github.shyiko.mysql.binlog.event.deserialization.AbstractRowsEventDataDeserializer","method":"deserializeRow","line":109},{"class":"com.github.shyiko.mysql.binlog.event.deserialization.WriteRowsEventDataDeserializer","method":"deserializeRows","line":64},{"class":"com.github.shyiko.mysql.binlog.event.deserialization.WriteRowsEventDataDeserializer","method":"deserialize","line":56},{"class":"com.github.shyiko.mysql.binlog.event.deserialization.WriteRowsEventDataDeserializer","method":"deserialize","line":32},{"class":"com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer","method":"deserializeEventData","line":337},{"class":"com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer","method":"nextEvent","line":246},{"class":"io.debezium.connector.binlog.BinlogStreamingChangeEventSource$1","method":"nextEvent","line":425},{"class":"com.github.shyiko.mysql.binlog.BinaryLogClient","method":"listenForEventPackets","line":1082},{"class":"com.github.shyiko.mysql.binlog.BinaryLogClient","method":"connect","line":657},{"class":"com.github.shyiko.mysql.binlog.BinaryLogClient$7","method":"run","line":959},{"class":"java.lang.Thread","method":"run","line":1583}]}}}}
{"timestamp":"2025-09-12T13:15:23.511365563Z","sequence":16416,"loggerClassName":"org.slf4j.impl.Slf4jLogger","loggerName":"io.debezium.connector.binlog.BinlogStreamingChangeEventSource","level":"WARN","message":"Error during binlog processing. Last offset stored = {ts_sec=1757682730, file=mariadb-bin.219995, pos=1114640, gtids=0-2420921427-2869908247, server_id=2420921427, event=4}, binlog reader near position = mariadb-bin.219995/1114684","threadName":"blc-adv-prod-online-test-srv01.duok5.tv:3306","threadId":46,"mdc":{"dbz.connectorLogicalName":"debezium","dbz.taskId":"0","dbz.connectorType":"MariaDB","dbz.connectorContext":"binlog"},"ndc":"","hostName":"64a025d9b9c0","processName":"/usr/lib/jvm/java-21-openjdk-21.0.8.0.9-1.el8.x86_64/bin/java","processId":1}
{code}
h1. Additional information related to database

binlog compression is disabled as well:
{code:java}
log_bin_compress | OFF
binlog_format | ROW
{code}
table schema:
MariaDB ******** SLAVE ******** READ ONLY ******** [web_test]> describe slip_log;
{+}---------{-}{{-}}{{-}}{{-}}{-}{{-}}{-}{-}{+}-------------{-}{{-}}{{-}}{{-}}{+}{{+}}{{-}}{{-}}{-}{{-}}{{-}}{{-}}{{-}}{{-}}{-}{{-}}{{-}}{+}{{+}}{{-}}{{-}}{-}---------------{-}{{-}}{{-}}{{-}}{{-}}{{-}}{{-}}{-}------------\{+}
|Field      |Type                |Null|Key|Default              |Extra          |

{+}---------{-}{{-}}{{-}}{{-}}{-}{{-}}{-}{-}{+}-------------{-}{{-}}{{-}}{{-}}{+}{{+}}{{-}}{{-}}{-}{{-}}{{-}}{{-}}{{-}}{{-}}{-}{{-}}{{-}}{+}{{+}}{{-}}{{-}}{-}---------------{-}{{-}}{{-}}{{-}}{{-}}{{-}}{{-}}{-}------------\{+}
|slip_log_id|bigint(20) unsigned|NO  |PRI|NULL                |auto_increment|
|ts          |timestamp(6)        |NO  |MUL|current_timestamp(6)|               |
|slip_uuid  |binary(16)          |NO  |MUL|NULL                |               |
|slip_id    |bigint(20) unsigned|NO  |   |NULL                |               |
|website_id  |bigint(20) unsigned|YES  |MUL|NULL                |               |
|marshaled  |mediumblob          |YES  |   |NULL                |               |

{+}---------{-}{{-}}{{-}}{{-}}{-}{{-}}{-}{-}{+}-------------{-}{{-}}{{-}}{{-}}{+}{{+}}{{-}}{{-}}{-}{{-}}{{-}}{{-}}{{-}}{{-}}{-}{{-}}{{-}}{+}{{+}}{{-}}{{-}}{-}---------------{-}{{-}}{{-}}{{-}}{{-}}{{-}}{{-}}{-}------------\{+}

Contributor guide

Open the contributing guide

Research direction

Start with the MariaDbConnector configuration and the BinlogStreamingChangeEventSource stack trace, then inspect EventDeserializer around the reported binlog position and table id 6276. Compare the stored offset with the preceding TABLE_MAP event; the issue does not state an expected behavior, so a concrete reproduction and acceptance condition are still needed.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, mariadb
Domain
backend, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.