debezium / debezium/dbz

Infinite Loop in Outbox Router when topic does not exists [DBZ-3652]

Open
#474 0 comments 0 reactions 0 assignees View on GitHub
component/outbox component/postgresql-connector migrated-from-jira type/bug
Dominant language
HTML
Stars
6
Forks
8
Avg merge
2d 19h
Merged PRs (30d)
1

Description

Migrated from [DBZ-3652](https://issues.redhat.com/browse/DBZ-3652)

Marked as a bug, but depends on what configuration means.

We are using AWS RDS as a Postgres database based on Postgres 11 version.
We are using *Debezium 1.4.1* version with *pgoutput* and *outbox router.* 
Recently, I've been testing what will happen if the topic will not be configured in Kafka.
Normally, in that case, connector login warns & stuck in an infinite loop as the task is unable to produce an event to Kafka:
{code:java}
 WARN [Producer clientId=connector-producer-piotrek-test-3.outbox_events_debezium_connector-0] Error while fetching metadata with correlation id 3884 : {piotrek-test-123.partners.feature-flag-events=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.NetworkClient)
{code}
So, in that case, even when I set up *event.processing.failure.handling.mode* still raises the errors and is stuck in the loop forever.

As an outbox router is configured to be able to fetch data from one table and route them to multiple different topics, one typo can block the entire queue from consuming & bloat databases as the connector will be stuck on a single event forever/until someone creates a topic. 

Connector configuration looks
{code:json}
{
"connector.class": "io.debezium.connector.postgresql.PostgresConnector", "transforms.outbox.type": "io.debezium.transforms.outbox.EventRouter", "slot.name": "debezium",
"tasks.max": "4",
"publication.name": "outbox_publication",
"transforms": "outbox",
"tombstones.on.delete": "false", "transforms.outbox.route.topic.replacement": "piotrek-test-3.${routedByValue}",
"publication.autocreate.mode": "disabled",
"database.user": "XXXX",
"database.dbname": "XXXX", "transforms.outbox.table.fields.additional.placement": "uuid:envelope:uuid,event_type:envelope:event_type,timestamp:envelope:timestamp",
"transforms.outbox.table.field.event.key": "partition_key", "transforms.outbox.table.field.event.timestamp": "timestamp", "database.server.name": "XXXX",
"transforms.outbox.table.field.event.id": "uuid", "transforms.outbox.route.by.field": "topic_name", "event.processing.failure.handling.mode": "skip",
"database.port": "5432",
"plugin.name": "pgoutput",
"topic.creation.enable": "false",
"database.hostname": "XXXX",
"database.password": "XXXX",
"name": "outbox_events_debezium_connector", "transforms.outbox.table.field.event.payload": "proto_payload", "table.include.list": "public.outbox_events",
"snapshot.mode": "exported", "transforms.outbox.table.field.event.payload.id": "partition_key"
}
{code}
 

 

Contributor guide

Open the contributing guide

Research direction

No source files or tests are named in the report. Start by reproducing the missing-topic case with the shown PostgreSQL/Debezium outbox-router configuration and event.processing.failure.handling.mode=skip; done means the connector no longer remains in an infinite loop when Kafka reports UNKNOWN_TOPIC_OR_PARTITION.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, java, kafka, postgresql
Domain
backend, databases, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.