debezium / debezium/dbz

Support to avoid circular fetching for PostgreSQL by pgoutput pulgin. [DBZ-3805]

Open
#495 5 comments 0 reactions 0 assignees View on GitHub
component/postgresql-connector migrated-from-jira type/enhancement
Dominant language
HTML
Stars
6
Forks
8
Avg merge
2d 19h
Merged PRs (30d)
1

Description

Migrated from [DBZ-3805](https://issues.redhat.com/browse/DBZ-3805)

# Question

Postgres connector not support capture the data that is just "write" to this db, except for the replicated data. so that If there are two bidirectional copies of DB, Debezium will capture a lot of useless data.

# Solution

As per the specification of the [Logical Replication Message Format](https://www.postgresql.org/docs/12/protocol-logicalrep-message-formats.html), the pgoutput plugin supports the 'Origin' message type to record the data origin name, for instance in the case of multi-master set-ups. So, if we come across this field and find that the source is something we want to avoid copying, we can skip it until we hit 'Commit'.

Another approach is to put the Origin name field in the message body and let the user decide whether the messages are processed or not.

Contributor guide

Open the contributing guide

Research direction

Start by reading the PostgreSQL Logical Replication Message Format specification linked in the issue, focusing on the pgoutput Origin message and transaction boundaries. Determine whether the intended behavior is filtering origins before Commit or exposing the origin for user decisions; the issue does not identify files, tests, or a final acceptance condition.

Written by the indexing model from the issue text.

Assessment

Tech stack
postgresql
Domain
databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.