apache / apache/pulsar

delay message deliver lost messages

Open
#15,171 10 comments 0 reactions 0 assignees View on GitHub
lifecycle/stale Stale type/bug
Dominant language
Java
Stars
15.3k
Forks
3.8k
Avg merge
1d 14h
Merged PRs (30d)
160

Description

**Describe the bug**
i am using pulsar's delay message feature,when i increate message count to 10000,consumer will only get 5000 messages or so ,other messages were lost,1000 delay message deliver worked well.

producer code:
SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
for(int i=0;i<10000;i++){
producer.createMessage("clicks2","hello world"+sdf.format(new Date()))
.deliverAfter(1L, TimeUnit.MINUTES).sendAsync();
}

consumer code:
private AtomicInteger count = new AtomicInteger(0);

@PulsarConsumer(topic="clicks2",subscriptionType= SubscriptionType.Shared,clazz=String.class)
public void consume(String msg) {
SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
log.info("msg:{},收到数据时间:{}",msg,sdf.format(new Date()));
int total = count.incrementAndGet();
log.info("总共收到数据量:{}",total);
}

Contributor guide

Open the contributing guide

Research direction

No repository files or tests are named. Start by reproducing the delay-message case with the provided Java producer and consumer, comparing 1,000 and 10,000 messages and recording delivery counts. Trace the delay-message delivery path to determine where messages disappear, then add a regression test showing that all scheduled messages are delivered.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.