[BUG] ThresholdShedder Bundles Inaccurate traffic statistics
- Dominant language
- Java
- Stars
- 15.3k
- Forks
- 3.8k
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 160
Description
broker DEBUG Log in leader :
2022-06-20 18:50:20.324 [pulsar-load-manager-1-1] INFO o.a.p.broker.loadbalance.impl.ThresholdShedder - [x.x.x.x:8080] broker is planning to shed throughput 0.0 MByte/s less than minimumThroughputThreshold 10.0 MByte/s, skipping bundle unload.
ZK STAT
get /loadbalance/broker-time-average/x.x.x.x:8080
{"shortTermMsgThroughputIn":3.8429246523532754E-20,"shortTermMsgThroughputOut":1.5286027972266306E-23,"shortTermMsgRateIn":8.361056820071499E-23,"shortTermMsgRateOut":2.9078053094894676E-26,"longTermMsgThroughputIn":2.51609898074917E7,"longTermMsgThroughputOut":75116.58169305474,"longTermMsgRateIn":54734.34121373813,"longTermMsgRateOut":34.42857810199835}
Other Server Stats look ok
{"shortTermMsgThroughputIn":1.955625028980416E7,"shortTermMsgThroughputOut":4.911981732451391E-26,"shortTermMsgRateIn":42499.15646885524,"shortTermMsgRateOut":7.995873567619153E-29,"longTermMsgThroughputIn":1.4583922183369141E7,"longTermMsgThroughputOut":2301.2118711286466,"longTermMsgRateIn":31706.082609610632,"longTermMsgRateOut":0.9013642705395061}
BUT
current server CPU of broker useage is 1300+%CPU and I am sure throughput in and out max than 10.0 MByte/s
Contributor guide
Research direction
Start with the broker DEBUG output from ThresholdShedder and inspect how the /loadbalance/broker-time-average/ ZK statistics are produced and consumed. Compare the short-term and long-term values with the broker's reported CPU and traffic, then verify that threshold-based bundle shedding uses accurate statistics and does not incorrectly skip unloading.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100