debezium / debezium/dbz

Debezium can break when two column names differs only in letter case

Open
#4 3 comments 0 reactions 1 assignee Claimed by @akasar9 View on GitHub
component/core-library component/postgresql-connector good first issue type/bug
Dominant language
HTML
Stars
6
Forks
8
Avg merge
2d 19h
Merged PRs (30d)
1

Description

I'm getting the exception below for two tables (out of 231 tables). In looking at the resolveColumnsFromStreamTupleData method, it seems that the short numberOfColumns = buffer.getShort(); is longer than the table.columns() which causes this exception and the connector shuts down. I was able get past the problem by adding the two offending tables to the table blacklist. This is all in development and no schema changes are being made to the tables after starting the connector.

The connector is being created by the Strimzi Kafka operator. Also note I'm using an apicurio schema registry and the avro converters.

{code}
ERROR Producer failure (io.debezium.pipeline.ErrorHandler) [debezium-postgresconnector-encdb-change-event-source-coordinator]
java.lang.IndexOutOfBoundsException: Index 299 out of bounds for length 299
at java.base/jdk.internal.util.Preconditions.outOfBounds(Preconditions.java:64)
at java.base/jdk.internal.util.Preconditions.outOfBoundsCheckIndex(Preconditions.java:70)
at java.base/jdk.internal.util.Preconditions.checkIndex(Preconditions.java:248)
at java.base/java.util.Objects.checkIndex(Objects.java:372)
at java.base/java.util.ArrayList.get(ArrayList.java:459)
at java.base/java.util.Collections$UnmodifiableList.get(Collections.java:1310)
at java.base/java.util.Collections$UnmodifiableList.get(Collections.java:1310)
at io.debezium.connector.postgresql.connection.pgoutput.PgOutputMessageDecoder.resolveColumnsFromStreamTupleData(PgOutputMessageDecoder.java:561)
at io.debezium.connector.postgresql.connection.pgoutput.PgOutputMessageDecoder.decodeInsert(PgOutputMessageDecoder.java:379)
at io.debezium.connector.postgresql.connection.pgoutput.PgOutputMessageDecoder.processNotEmptyMessage(PgOutputMessageDecoder.java:191)
at io.debezium.connector.postgresql.connection.AbstractMessageDecoder.processMessage(AbstractMessageDecoder.java:42)
at io.debezium.connector.postgresql.connection.PostgresReplicationConnection$1.deserializeMessages(PostgresReplicationConnection.java:480)
at io.debezium.connector.postgresql.connection.PostgresReplicationConnection$1.readPending(PostgresReplicationConnection.java:472)
at io.debezium.connector.postgresql.PostgresStreamingChangeEventSource.searchWalPosition(PostgresStreamingChangeEventSource.java:271)
at io.debezium.connector.postgresql.PostgresStreamingChangeEventSource.execute(PostgresStreamingChangeEventSource.java:131)
at io.debezium.pipeline.ChangeEventSourceCoordinator.streamEvents(ChangeEventSourceCoordinator.java:140)
at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:113)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
{code}

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.