debezium / debezium/dbz

Pause/Resume specific incremental snapshots [DBZ-6139]

Open
#769 0 comments 0 reactions 0 assignees View on GitHub
component/core-library migrated-from-jira type/enhancement
Dominant language
HTML
Stars
6
Forks
9
Avg merge
2d 19h
Merged PRs (30d)
1

Description

Migrated from [DBZ-6139](https://issues.redhat.com/browse/DBZ-6139)

The {{pause-snapshot}} signal's current behavior is to place the entire Incremental Snapshot process into a paused state until it is later resumed.

Where Debezium may be deployed in a XaaS that supports triggering Incremental Snapshots (or even Debezium UI users), it could be helpful where we allow pausing specific snapshots without necessarily enforcing a pause-all / resume-all approach.

I could see two business units or teams that have submitted incremental snapshots that focus on different tables or perhaps different subsets of data from the same table, and there could be a reason to pause the first signal received in favor of the next signal. Maybe the first signal includes several million rows where the latter is a specific record and the user doesn't want to wait for the former to finish or cancel the former signal to restart it later.

This introduces some challenges because, currently, the Incremental Snapshot process acts much like a queue of collections that get processed in the order the signals are received. To support this notion, we would have to reimagine how we process this queue allowing elements that are later in the queue to be processed first if earlier elements are "paused".

Additionally, we'll need to also amend the resume functionality to pin-point the specific paused elements in the queue and resume those.

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the incremental snapshot process described in the issue, including the pause-snapshot and resume functionality and the queue of collections processed from received signals. Done would mean defining and implementing a way to pause and resume specific queued snapshots while allowing later snapshots to proceed.

Written by the indexing model from the issue text.

Assessment

Domain
databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.