apache / apache/pulsar

[Bug] Java consumer blocked by message larger than batchReceivePolicy.maxNumBytes when calling batchReceiveAsync

Open
#21,631 5 comments 0 reactions 2 assignees Claimed by @Technoboy- View on GitHub
type/bug
Dominant language
Java
Stars
15.3k
Forks
3.8k
Avg merge
1d 14h
Merged PRs (30d)
160

Description

### Search before asking

- [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.

### Version

Java SDK Version: 2.7.4
OS: Linux

### Minimal reproduce step

1. create a producer with compressionType on
2. create a consumer and call `batchReceiveAsync` with default `batchReceivePolicy` which has 10MB `maxSizeOfMessages`
3. On the producer side, we send a message with larger than 10MB whose compressed size should be less than broker limit (5MB by default), so the message can be sent successfully.

### What did you expect to see?

the large message should be consumed and received in the callback of `batchReceivePolicy`

### What did you see instead?

the large message never shown up in the callback, same for message sent after the large message. Increasing `maxNumBytes` to be larger than the message size can solve the problem.

### Anything else?

Looking at the SDK code, we find if any message in `incommingMessages` queue is larger than `maxNumBytes` (10MB by default), it will block this consumer unless the message is expired. So I wonder if this is the expected behavior and why?

image
image

### Are you willing to submit a PR?

- [X] I'm willing to submit a PR!

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.