[Bug] Permit calculations are mixing individual message and batch message counts, similar issue in documentation
- Dominant language
- Java
- Stars
- 15.3k
- Forks
- 3.8k
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 160
Description
### Search before asking
- [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.
### Read release policy
- [X] I understand that unsupported versions don't get bug fixes. I will attempt to reproduce the issue on a supported version of Pulsar client and Pulsar broker.
### Version
master
### Minimal reproduce step
Fixing some issues as part of #23231 such as
https://github.com/apache/pulsar/blob/14395295b4996dcfb7eac288d92baf1104c9c576/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentDispatcherMultipleConsumers.java#L790-L791
The problem is that permits use a batch message count and unack messages uses individual message counts. These shouldn't be mixed. It's possible to convert the batch message count to an estimated number of messages with the average number of messages in batch value that is kept in the consumer.
There are several locations where this is mixed up.
This also applies to documentation. The javadoc of receiverQueueSize is very confusing and doesn't tell that it's counted in batch messages: https://github.com/apache/pulsar/blob/10f4e0248f0f985b1dc7ad38970c906b7fe629be/pulsar-client-api/src/main/java/org/apache/pulsar/client/api/ConsumerBuilder.java#L356-L387
### What did you expect to see?
The permit calculations in dispatchers shouldn't mix individual message counts and batch message counts.
The documentation needs updates too so that it's clearly defined whether a "message" refers to an individual message or a batch message (a batch of individual messages).
### What did you see instead?
- calculations mixed in several locations
- documentation doesn't clearly define whether a message is an individual message or a batch of messages. example is the receiverQueueSize javadoc.
### Anything else?
_No response_
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
Contributor guide
Research direction
Start with PersistentDispatcherMultipleConsumers.java at the linked permit calculation and inspect the other locations where individual and batch message counts are mixed. Then read the receiverQueueSize Javadoc in pulsar-client-api/src/main/java/org/apache/pulsar/client/api/ConsumerBuilder.java; done means calculations use consistent count semantics and the documentation clearly distinguishes individual messages from batches.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100