apache / apache/pulsar

Provide DLQ capibilities for Sinks

Open
#11,444 1 comment 0 reactions 0 assignees View on GitHub
lifecycle/stale type/enhancement
Dominant language
Java
Stars
15.3k
Forks
3.8k
Avg merge
1d 14h
Merged PRs (30d)
160

Description

Currently Sinks do not have DLQ capabilities. There are many cases for which this would be useful. Eg. the message is not suitable for the sink destination.

Functions already have this capability, exposed via `deadLetterTopic` and `maxMessageRetries`.
It would be useful for the sinks API to support this too.

Contributor guide

Open the contributing guide

Research direction

Start with the Sinks API and inspect how Functions expose the existing `deadLetterTopic` and `maxMessageRetries` capabilities. Determine the corresponding sink configuration and behavior needed for messages unsuitable for the sink destination; done means sinks support an equivalent DLQ flow and retry limit.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend-api-design, distributed-systems
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.