debezium / debezium/dbz

PostgreSQL ReadOnly Incremental Snapshots [DBZ-5046]

Open
#623 0 comments 0 reactions 0 assignees View on GitHub
migrated-from-jira type/enhancement
Dominant language
HTML
Stars
6
Forks
8
Avg merge
2d 19h
Merged PRs (30d)
1

Description

Migrated from [DBZ-5046](https://issues.redhat.com/browse/DBZ-5046)

h1. Feature request or enhancement

For feature requests or enhancements, provide this information, please:
h2. Which use case/requirement will be addressed by the proposed feature?

This proposal would remove the requirement for a signaling table in Postgres to trigger Incremental Snapshots. I also wonder it this might allow Incremental Snapshots to be the default snapshot mode for the initial snapshot.
h2. Implementation ideas (optional)

Background:
* Postgres has a `pg_logical_emit_message()` command which allows arbitrary messages to be passed through the logical decoding chain. 
* These logical decoding messages have always been supported by `wal2json` and are supported by `pgoutput` since Postgres 14, however, before Postgres 14 `pgoutput` would not support this signaling mechanism.
* https://www.postgresql.org/docs/current/functions-admin.html

Implementation:
* Replace the high and low watermarks in signaling table with messages emitted using `pg_logical_emit_message()`.
* Replace `execute-snapshot` message in signaling table with message to `pg_logical_emit_message()`.
*  

Contributor guide

Open the contributing guide

Research direction

The issue names no repository files or tests. Start with PostgreSQL documentation for pg_logical_emit_message(), then inspect the existing incremental snapshot signaling implementation. Done means evaluating replacement of signaling-table watermarks and execute-snapshot messages, including pgoutput support before PostgreSQL 14 and whether this enables the proposed default snapshot mode.

Written by the indexing model from the issue text.

Assessment

Tech stack
postgresql
Domain
databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.