apache / apache/rocketmq-flink

Some problems found by individuals during use

Open
#78 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
174
Forks
104
PR merge metrics
No merged PRs in 30d

Description

**1. If you enable a new topic and a new tag, it seems that an exception will be thrown when the project starts**

> java.lang.RuntimeException: org.apache.rocketmq.client.exception.MQClientException: The message queue is not in assigned list, may be rebalancing, message queue: MessageQueue [topic=TEST_FLINK_TOPIC5, brokerName=DKF-RMQ-BK-1, queueId=2]

**2. It seems that the offset of Rocketmq is not used correctly**

**_Producer Log_**

> 16:29:24.738 [Thread-0] INFO com.buk.flink.rocketmq2.MqProducer - 【RocketMq 生产者】成功!content:Test Message 0,sendResult:SendResult [sendStatus=SEND_OK, msgId=7F0000014A9018B4AAC298E241760000, offsetMsgId=C0A81AC800002A9F0000000089E1D5FE, messageQueue=MessageQueue [topic=TEST_FLINK_TOPIC5, brokerName=DKF-RMQ-BK-1, queueId=0], queueOffset=21]

**_Consumer log_**
Consumers will receive many identical messages

> 16:29:48.439 [rmq-pull-thread-0] DEBUG o.a.r.f.l.RocketMQSourceFunction - 7F0000014A9018B4AAC298E241760000_DKF-RMQ-BK-1 0 21
16:29:48.439 [rmq-pull-thread-0] INFO c.b.f.r.FlinkMqDataStreamConsumer - 收到消息:Test Message 0
16:29:48.439 [rmq-pull-thread-0] DEBUG o.a.r.f.l.RocketMQSourceFunction - 7F0000014A9018B4AAC298E241760000_DKF-RMQ-BK-1 0 21
16:29:48.439 [rmq-pull-thread-0] INFO c.b.f.r.FlinkMqDataStreamConsumer - 收到消息:Test Message 0
16:29:48.439 [rmq-pull-thread-0] DEBUG o.a.r.f.l.RocketMQSourceFunction - 7F0000014A9018B4AAC298E241760000_DKF-RMQ-BK-1 0 21
16:29:48.439 [rmq-pull-thread-0] INFO c.b.f.r.FlinkMqDataStreamConsumer - 收到消息:Test Message 0

**_After commenting out the following code, it will be normal_**
![image](https://user-images.githubusercontent.com/38726443/204750897-d5e6ab7b-09cf-4594-bfb8-f4287a00c1f4.png)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the startup exception and duplicate-message behavior from the reported logs. Inspect RocketMQSourceFunction and FlinkMqDataStreamConsumer, then compare the behavior with the code shown in the issue commented out. Done means enabling a new topic and tag does not fail at startup and the consumer does not receive repeated messages.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
stream-processing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.