apache / apache/pulsar-client-go
aynsc send [hasSpace()] control has bad performance when send large message
- Dominant language
- Go
- Stars
- 745
- Forks
- 389
- Avg merge
- 3d 20h
- Merged PRs (30d)
- 3
Description
#### Expected behavior
1. when i send large message body,the client has a queue to hold it and send by batch, there are two triggers: 1. message
count lager then `maxMessages`, OR 2. messge body sum is larger then 'BatchingMaxSize'
2. Know the codes are 【bc.numMessages+1 <= bc.maxMessages && bc.buffer.ReadableBytes()+msgSize <= uint32(bc.maxBatchSize)】,and it will brings somemany Flush opertion when i send large message body
#### Actual behavior
1. 【unable to add message to batch】will called when send message body size nearly 'BatchingMaxSize'
#### Steps to reproduce
1. send body size lager then 3M
2. make send fast than 1000 qps
#### System configuration
**Pulsar version**: 2.9.0
Contributor guide
Research direction
Start at the async send path's hasSpace() control and inspect the batching condition involving maxMessages, BatchingMaxSize, buffer.ReadableBytes(), and Flush operations. Reproduce with messages larger than 3 MB at over 1000 QPS, then verify that batching avoids the reported “unable to add message to batch” behavior and unnecessary flushes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100