debezium / debezium/dbz

MySQL CDC connector failing with java.lang.ArrayIndexOutOfBoundsException [DBZ-2430]

Open
#355 0 comments 0 reactions 0 assignees View on GitHub
migrated-from-jira type/bug
Dominant language
HTML
Stars
6
Forks
8
Avg merge
2d 19h
Merged PRs (30d)
1

Description

Migrated from [DBZ-2430](https://issues.redhat.com/browse/DBZ-2430)

I'm trying to on-board a new MySQL connector on debezium 0.9.5 Final. It works fine initially, then breaks as soon as there is an insert / update operation on the table with the following exception:

 {code}
[2020-08-06 15:18:27,345] ERROR MySQL|PAYMENTS_SDMONEY|binlog Failed to properly convert data value for 'payments_sdmoney.t_MoneyOutTransaction.sdIdentity' of type VARCHAR for row [[70, 65, 73, 76, 69, 68, 95, 79, 78, 95, 77, 65, 78, 85, 65, 76, 95, 82, 69, 67, 7`9, 78], 2020-08-06T15:18:27, [82, 69, 74, 69, 67, 84, 69, 68], [67, 65, 48, 51, 51, 51, 55, 49, 56, 50, 52, 49], [73, 110, 99, 111, 114, 114, 101, 99, 116, 32, 98, 1
01, 110, 101, 102, 105, 99, 105, 97, 114, 121, 32, 97, 99, 99, 111, 117, 110, 116, 32, 100, 101, 116, 97, 105, 108]]: (io.debezium.relational.TableSchemaBuilder)
java.lang.IllegalArgumentException: Unexpected value for JDBC type 12 and column sdIdentity VARCHAR(64) CHARSET utf8 NOT NULL: class=class java.time.LocalDateTime
at io.debezium.jdbc.JdbcValueConverters.handleUnknownData(JdbcValueConverters.java:1164)
at io.debezium.jdbc.JdbcValueConverters.convertValue(JdbcValueConverters.java:1207)
at io.debezium.connector.mysql.MySqlValueConverters.convertString(MySqlValueConverters.java:364)
at io.debezium.connector.mysql.MySqlValueConverters.lambda$converter$13(MySqlValueConverters.java:277)
at io.debezium.connector.mysql.MySqlValueConverters$$Lambda$385/496105738.convert(Unknown Source)
at io.debezium.relational.TableSchemaBuilder.lambda$createValueGenerator$2(TableSchemaBuilder.java:229)
at io.debezium.relational.TableSchemaBuilder$$Lambda$389/1179627047.apply(Unknown Source)
at io.debezium.relational.TableSchema.valueFromColumnData(TableSchema.java:145)
at io.debezium.connector.mysql.RecordMakers$1.update(RecordMakers.java:263)
at io.debezium.connector.mysql.RecordMakers$RecordsForTable.update(RecordMakers.java:481)
at io.debezium.connector.mysql.BinlogReader.handleUpdate(BinlogReader.java:876)
at io.debezium.connector.mysql.BinlogReader$$Lambda$445/904297834.accept(Unknown Source)
at io.debezium.connector.mysql.BinlogReader.handleEvent(BinlogReader.java:492)
at io.debezium.connector.mysql.BinlogReader$$Lambda$396/555823683.onEvent(Unknown Source)
at com.github.shyiko.mysql.binlog.BinaryLogClient.notifyEventListeners(BinaryLogClient.java:1095)
at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:943)
at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:580)
at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:825)
at java.lang.Thread.run(Thread.java:745)
[2020-08-06 15:18:27,345] ERROR MySQL|PAYMENTS_SDMONEY|binlog Failed to properly convert data value for 'payments_sdmoney.t_MoneyOutTransaction.amount' of type DECIMAL for row [[70, 65, 73, 76, 69, 68, 95, 79, 78, 95, 77, 65, 78, 85, 65, 76, 95, 82, 69, 67, 79, 78], 2020-08-06T15:18:27, [82, 69, 74, 69, 67, 84, 69, 68], [67, 65, 48, 51, 51, 51, 55, 49, 56, 50, 52, 49], [73, 110, 99, 111, 114, 114, 101, 99, 116, 32, 98, 101,
110, 101, 102, 105, 99, 105, 97, 114, 121, 32, 97, 99, 99, 111, 117, 110, 116, 32, 100, 101, 116, 97, 105, 108]]: (io.debezium.relational.TableSchemaBuilder)
java.lang.IllegalArgumentException: Unexpected value for JDBC type 3 and column amount DECIMAL(8, 2) NOT NULL: class=class [B
at io.debezium.jdbc.JdbcValueConverters.handleUnknownData(JdbcValueConverters.java:1164)
at io.debezium.jdbc.JdbcValueConverters.convertValue(JdbcValueConverters.java:1207)
at io.debezium.jdbc.JdbcValueConverters.toBigDecimal(JdbcValueConverters.java:926)
at io.debezium.jdbc.JdbcValueConverters.convertDecimal(JdbcValueConverters.java:918)
at io.debezium.jdbc.JdbcValueConverters.lambda$converter$11(JdbcValueConverters.java:299)
at io.debezium.jdbc.JdbcValueConverters$$Lambda$387/1990090759.convert(Unknown Source)
at io.debezium.relational.TableSchemaBuilder.lambda$createValueGenerator$2(TableSchemaBuilder.java:229)
at io.debezium.relational.TableSchemaBuilder$$Lambda$389/1179627047.apply(Unknown Source)
at io.debezium.relational.TableSchema.valueFromColumnData(TableSchema.java:145)
at io.debezium.connector.mysql.RecordMakers$1.update(RecordMakers.java:263)
at io.debezium.connector.mysql.RecordMakers$RecordsForTable.update(RecordMakers.java:481)
at io.debezium.connector.mysql.BinlogReader.handleUpdate(BinlogReader.java:876)
at io.debezium.connector.mysql.BinlogReader$$Lambda$445/904297834.accept(Unknown Source)
at io.debezium.connector.mysql.BinlogReader.handleEvent(BinlogReader.java:492)
at io.debezium.connector.mysql.BinlogReader$$Lambda$396/555823683.onEvent(Unknown Source)
at com.github.shyiko.mysql.binlog.BinaryLogClient.notifyEventListeners(BinaryLogClient.java:1095)
at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:943)
at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:580)
at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:825)
at java.lang.Thread.run(Thread.java:745)
[2020-08-06 15:18:27,345] ERROR MySQL|PAYMENTS_SDMONEY|binlog Error during binlog processing. Last offset stored = null, binlog reader near position = mysql-bin.006484/50981202 (io.debezium.connector.mysql.BinlogReader)
[2020-08-06 15:18:27,345] ERROR MySQL|PAYMENTS_SDMONEY|binlog Failed due to error: Error processing binlog event (io.debezium.connector.mysql.BinlogReader)
org.apache.kafka.connect.errors.ConnectException
at io.debezium.connector.mysql.AbstractReader.wrap(AbstractReader.java:230)
at io.debezium.connector.mysql.AbstractReader.failed(AbstractReader.java:208)
at io.debezium.connector.mysql.BinlogReader.handleEvent(BinlogReader.java:508)
at io.debezium.connector.mysql.BinlogReader$$Lambda$396/555823683.onEvent(Unknown Source)
at com.github.shyiko.mysql.binlog.BinaryLogClient.notifyEventListeners(BinaryLogClient.java:1095)
at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:943)
at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:580)
at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:825)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ArrayIndexOutOfBoundsException
[2020-08-06 15:18:27,345] INFO MySQL|PAYMENTS_SDMONEY|binlog Error processing binlog event, and propagating to Kafka Connect so it stops this connector. Future binlog events read before connector is shutdown will be ignored. (io.debezium.connector.mysql.BinlogReader)
[2020-08-06 15:18:27,661] INFO || WorkerSourceTask\{id=SOURCE-CDC-PAYMENTS_SDMONEY_NEW_2-0} Committing offsets (org.apache.kafka.connect.runtime.WorkerSourceTask)
[2020-08-06 15:18:27,661] INFO || WorkerSourceTask\{id=SOURCE-CDC-PAYMENTS_SDMONEY_NEW_2-0} flushing 0 outstanding messages for offset commit (org.apache.kafka.connect.runtime.WorkerSourceTask)
[2020-08-06 15:18:27,662] ERROR || WorkerSourceTask\{id=SOURCE-CDC-PAYMENTS_SDMONEY_NEW_2-0} Task threw an uncaught and unrecoverable exception (org.apache.kafka.connect.runtime.WorkerTask)
org.apache.kafka.connect.errors.ConnectException
at io.debezium.connector.mysql.AbstractReader.wrap(AbstractReader.java:230)
at io.debezium.connector.mysql.AbstractReader.failed(AbstractReader.java:208)
at io.debezium.connector.mysql.BinlogReader.handleEvent(BinlogReader.java:508)
at io.debezium.connector.mysql.BinlogReader$$Lambda$396/555823683.onEvent(Unknown Source)
at com.github.shyiko.mysql.binlog.BinaryLogClient.notifyEventListeners(BinaryLogClient.java:1095)
at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:943)
at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:580)
at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:825)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ArrayIndexOutOfBoundsException
[2020-08-06 15:18:27,662] ERROR || WorkerSourceTask\{id=SOURCE-CDC-PAYMENTS_SDMONEY_NEW_2-0} Task is being killed and will not recover until manually restarted (org.apache.kafka.connect.runtime.WorkerTask)
{code}

Contributor guide

Open the contributing guide

Research direction

Start with the reported MySQL CDC failure on Debezium 0.9.5 and reproduce it using an insert or update on the affected table. Trace the stack through MySqlValueConverters.java, JdbcValueConverters.java, TableSchemaBuilder.java, and RecordMakers.java. Done means the connector processes the affected VARCHAR and DECIMAL values without stopping on the binlog event.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, mysql
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.