debezium / debezium/dbz

Check LSN before start CDC for a table [DBZ-4264]

Open
#528 0 comments 0 reactions 0 assignees View on GitHub
component/sqlserver-connector migrated-from-jira type/enhancement
Dominant language
HTML
Stars
6
Forks
8
Avg merge
2d 19h
Merged PRs (30d)
1

Description

Migrated from [DBZ-4264](https://issues.redhat.com/browse/DBZ-4264)

Some customers are willing to see more reliability/robustness in Debezium for SQL Server. 

What they are looking for is not capturing somehow “record x” in the following scenario, but is having an option to fail the connector with an error message.

 
{noformat}
Start connector
Disable capture
Insert a record x
Enable capture (Min LSN is NULL)
Insert a record y
{noformat}
 

In the end, record x is lost, which makes sense, but they would like instead to be aware of this out-of-sync issue and they would prefer instead the connector to fail with an error so that they can act accordingly.

 

Potential solution is to check LSN before start CDC for a table

IF @LastReadLSN < sys.fn_cdc_get_min_lsn('dbo_Accounts')
BEGIN
RAISERROR('The last read LSN is smaller than the minimum LSN in the Change Data Capture table.',16,1)
END

Contributor guide

Open the contributing guide

Research direction

Start with the reported SQL Server CDC scenario and the proposed comparison between @LastReadLSN and sys.fn_cdc_get_min_lsn('dbo_Accounts'). Trace where the connector starts CDC for a table and how it reports connector errors. Done means an out-of-sync LSN causes a clear failure instead of silently losing record x, with coverage for the described sequence.

Written by the indexing model from the issue text.

Assessment

Tech stack
sql
Domain
databases
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.