apache / apache/pulsar

FlinkPulsarSink EXACTLY_ONCE issue

Open
#16,037 1 comment 0 reactions 0 assignees View on GitHub
Stale type/bug
Dominant language
Java
Stars
15.3k
Forks
3.8k
Avg merge
1d 14h
Merged PRs (30d)
160

Description

**Describe the bug**
FlinkPulsarSink encounter issue with PulsarSinkSemantic.EXACTLY_ONCE.

**To Reproduce**
Steps to reproduce the behavior:
1. In apache-pulsar/conf/broker.conf, set transactionCoordinatorEnabled=true
2. Restart the broker
3. Added PulsarSinkSemantic.EXACTLY_ONCE
![image](https://user-images.githubusercontent.com/107404257/173338060-10804fdc-4b28-4e27-8178-6d5d9879ab21.png)
4. Encountered error Namespace not found, similar to
#15996
Here is a screenshot of my error:
![image](https://user-images.githubusercontent.com/107404257/173338235-dacf8b96-2d09-486c-92cf-551051e40d63.png)
5. Ran command:
sudo apache-pulsar/bin/pulsar initialize-transaction-coordinator-metadata -c pulsar-cluster-1 -cs 192.168.3.1:2181
6. And encountered another error: Transaction coordinator with id 1 not found.

**Desktop**
- ubuntu 20, pulsar 2.10.0, kop 2.10, flink 1.12

Contributor guide

Open the contributing guide

Research direction

Start with the transactionCoordinatorEnabled setting in apache-pulsar/conf/broker.conf and compare this report with issue #15996. Reproduce the EXACTLY_ONCE setup and the initialize-transaction-coordinator-metadata command using the reported environment. Done means the sink initializes without Namespace not found or Transaction coordinator with id 1 not found errors.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.