debezium / debezium/dbz

Snapshot multi-thread connections may lead to EOFException issues [DBZ-7932]

Open
#989 0 comments 0 reactions 0 assignees View on GitHub
component/core-library migrated-from-jira type/bug
Dominant language
HTML
Stars
6
Forks
8
Avg merge
2d 19h
Merged PRs (30d)
1

Description

Migrated from [DBZ-7932](https://issues.redhat.com/browse/DBZ-7932)

When configuring the connector with the following configuration:

{noformat}
snapshot.max.threads=5
table.include.list=T1,T2,T3,T4,T5,T6
{noformat}

If one table takes a long time to snapshot while the other tables only take a few minutes, all but one of the snapshot connections sit idle for extended periods of time, which can lead to problems when closing the connections or performing the rollback on the original JDBC connection at the end of the snapshot, especially if there are load balancers or other network configurations at play that break idle connections with the database.

This ultimately can cause an incomplete snapshot, although all data had been sent successfully.

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the configuration with snapshot.max.threads=5 and table.include.list=T1,T2,T3,T4,T5,T6, using a long-running table snapshot and idle connections. Trace connection closing and rollback on the original JDBC connection, including EOFException failures; done means the snapshot completes even when idle connections are broken by network infrastructure.

Written by the indexing model from the issue text.

Assessment

Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.