Changing isolation mode not closing old connections [DBZ-9486]
- Dominant language
- HTML
- Stars
- 6
- Forks
- 8
- Avg merge
- 2d 19h
- Merged PRs (30d)
- 1
Description
Migrated from [DBZ-9486](https://issues.redhat.com/browse/DBZ-9486)
In order to make your issue reports as actionable as possible, please provide the following information, depending on the issue type.
h1. Bug report
For bug reports, provide this information, please:
h2. What Debezium connector do you use and what version?
SQL Server 2.3.7
h2. What is the connector configuration?
{
"version": "2.3.7.4",
"connector.class": "io.debezium.connector.sqlserver.SqlServerConnector",
"database.server.name": "server",
"topic.prefix": "prefix",
"database.names": "DBName",
"database.port": "port",
"database.hostname": "host",
"database.user": "user",
"database.password": "pw",
"database.encrypt": "true",
"database.trustServerCertificate": true,
"include.schema.changes": "false",
"column.exclude.list": "columns"
"signal.data.collection": "debezium_signal",
"schema.history.internal": "schemahistory",
"schema.history.internal.file.filename": "filename",
"database.history": "databasehistory",
"database.history.file.filename": "filename",
"offset.flush.timeout.ms" : "10000",
"offset.flush.interval.ms": "100000",
"predicates": "isNotTombstone",
"predicates.isNotTombstone.type": "org.apache.kafka.connect.transforms.predicates.RecordIsTombstone",
"predicates.isNotTombstone.negate": "true",
"transforms": "reroute_api,reroute_dbo,unwrap,InsertKey,InsertValue,dates",
"transforms.reroute_api.type": "org.apache.kafka.connect.transforms.RegexRouter",
"transforms.reroute_api.regex": "replacement_(.*)\\.api\\.(.*)",
"transforms.reroute_api.replacement": "replacement_api_$2",
"transforms.reroute_dbo.type": "org.apache.kafka.connect.transforms.RegexRouter",
"transforms.reroute_dbo.regex": "replacement_(.*)\\.dbo\\.(.*)",
"transforms.reroute_dbo.replacement": "replacement_$2",
"transforms.unwrap.type": "io.debezium.transforms.ExtractNewRecordState",
"transforms.unwrap.delete.handling.mode": "rewrite",
"transforms.unwrap.add.fields": "op,ts_ms",
"transforms.unwrap.drop.tombstones": "false",
"transforms.InsertKey.type": "org.apache.kafka.connect.transforms.InsertField$Key",
"transforms.InsertKey.static.field": "EnterpriseId",
"transforms.InsertKey.static.value": "\{EnterpriseId}",
"transforms.InsertValue.type": "org.apache.kafka.connect.transforms.InsertField$Value",
"transforms.InsertValue.static.field": "EnterpriseId",
"transforms.InsertValue.static.value": "\{EnterpriseId}",
"transforms.dates.type": "com..kafka.connect.transform.ConvertFieldToUtc$Value",
"transforms.dates.timezone": "\{EnterpriseTimezone}",
"snapshot.mode": "initial",
"snapshot.isolation.mode": "snapshot",
"incremental.snapshot.chunk.size": "100000",
"table.include.list": "\{table.include.list}",
"errors.max.retries": "10"
}
h2. What is the captured database version and mode of deployment?
(E.g. on-premises, with a specific cloud provider, etc.)
SQL Server 13.0.4259.0(2016) - 16.0.4212.1(2022)
h2. What behavior do you expect?
When changing snapshot.isolation.mode from the default to snapshot the old connections are closed before new connections are created
h2. What behavior do you see?
We updated our debezium connectors configurations from the default to
{{"snapshot.isolation.mode": "snapshot"}}
{{}}
To do this we
# stopped the connectors
# updated the configurations
# started the connectors
When we did that the connector created new connections to the DB, but did not close the old ones. This resulted in 4 connections. Additionally, because those old transactions were left open and the DB was set to READ_COMMITTED_SNAPSHOT isolation it caused the Version Store to grow until the Temp DB was filled. We had to go through and kill those old connections to get the databases usable again.
h2. Do you see the same behaviour using the latest released Debezium version?
(Ideally, also verify with latest Alpha/Beta/CR version)
Have not tested
h2. Do you have the connector logs, ideally from start till finish?
(You might be asked later to provide [DEBUG/TRACE|https://debezium.io/documentation/reference/stable/operations/logging.html] level log)
No, we did not notice the issue for several days.
h2. How to reproduce the issue using our [tutorial|https://github.com/debezium/debezium-examples/tree/main/tutorial] deployment?
h1. Feature request or enhancement
For feature requests or enhancements, provide this information, please:
h2. Which use case/requirement will be addressed by the proposed feature?
h2. Implementation ideas (optional)
Contributor guide
Research direction
Start by reproducing the configuration change from the report: stop the SQL Server connector, change snapshot.isolation.mode to snapshot, and restart it. Observe the connector's database connections and open transactions before and after the change; done means old connections are closed before new ones are created and no lingering transactions cause Version Store growth.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- sql
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100