apache / apache/pulsar

[Bug] [broker]Disk iops cannot be higher than 4k,seems to be affected by Direct I/O

Open
#20,969 2 comments 0 reactions 0 assignees View on GitHub
Stale type/bug
Dominant language
Java
Stars
15.3k
Forks
3.8k
Avg merge
1d 14h
Merged PRs (30d)
160

Description

### Search before asking

- [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.

### Version

3 Aliyun ecs,broker and bookie share a disk
disk have 16800 iops
pulsar-version:3.0.0
CentOS Linux release 7.9.2009 (Core)
3.10.0-1160.88.1.el7.x86_64

### Minimal reproduce step

Through the fni tool, when testing disk performance, it is found that turning on/off direct has a great impact on disk iops
![image](https://github.com/apache/pulsar/assets/134363858/d89daf1c-ecbc-436b-a349-7a50179444e1)

### What did you expect to see?

I hope to see that the disk io of pulsar can exceed 4k

### What did you see instead?

![image](https://github.com/apache/pulsar/assets/134363858/4d93e952-3ddc-4789-9525-8465e9951bf2)

### Anything else?

_No response_

### Are you willing to submit a PR?

- [ ] I'm willing to submit a PR!

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the reported disk I/O behavior on Pulsar 3.0.0 with the broker and bookie sharing an Aliyun ECS disk, comparing direct I/O enabled and disabled. Check the broker and bookie disk-I/O paths to identify why throughput appears capped near 4k, then verify the result with the same disk-performance test.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.