debezium / debezium/dbz

Postgres connector consumes all connections when restarting [DBZ-9537]

Open
#94 0 comments 0 reactions 0 assignees View on GitHub
component/postgresql-connector migrated-from-jira type/bug
Dominant language
HTML
Stars
6
Forks
9
Avg merge
2d 19h
Merged PRs (30d)
1

Description

Migrated from [DBZ-9537](https://issues.redhat.com/browse/DBZ-9537)

In order to make your issue reports as actionable as possible, please provide the following information, depending on the issue type.
h1. Bug report

When replication slot is lost, connector attempts to restart. If errors.max.retries config isn't set connector will always restart. Debezium postgres task has two postgres connections in normal condition. However when connector's task is restarted due to replication slot's loss, the connections don't close. They stay in idle state (you can see it in pg_stat_activity). When task restart's it fails again due to lost slot and  leaves 2 new connections in idle state. It continues further and finally, debezium connector consumes all  available connections. After that task goes to failed status, but all connections stays in idle status and aren't closed even if  idle_session_timeout is configured.
h2. What Debezium connector do you use and what version?

2.6.2-Final
h2. What is the connector configuration?
h2. What is the captured database version and mode of deployment?

Postgres 16
h2. What behavior do you expect?

Connections of previous task are closed when it restarts
h2. What behavior do you see?

Connections stay in idle status
h2. Do you see the same behaviour using the latest released Debezium version?

Didn't check
h2. Do you have the connector logs, ideally from start till finish?

(You might be asked later to provide [DEBUG/TRACE|https://debezium.io/documentation/reference/stable/operations/logging.html] level log)

I'll provide it if required.
h2. How to reproduce the issue using our [tutorial|https://github.com/debezium/debezium-examples/tree/main/tutorial] deployment?

setup low max slot wal keep size for postgres, disable heartbeat table, execute queries on table of same db, which isn't tracked by debezium. Wait till replication slot gets lost. Observe consuming connections.

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the restart scenario from the debezium-examples/tutorial deployment with the described low WAL keep size and disabled heartbeat table, then observe pg_stat_activity during replication-slot loss. Done means previous-task connections no longer remain idle after restarts and the connector does not exhaust available PostgreSQL connections.

Written by the indexing model from the issue text.

Assessment

Tech stack
postgresql
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.