IOPS was too high when mq was cleanning up expired files automatically
- Dominant language
- Java
- Stars
- 22.6k
- Forks
- 12k
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 26
Description
### Search before creation
- [x] I had searched in the [issues](https://github.com/apache/rocketmq/issues) and found no similar issues.
### Documentation Related
Cluster's flush type is SYNC;
Cluster's deployment type is 2m-2s;
Cluster's version is 4.9.7;
300-500 topics in cluster;
And other runtime settings are almost default.
I found the IOPS was too high when deleted expired files automatically in CommitLog, IndexFile and ConsumeQueue.
What reasons could lead to this? How to decrease IO.
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
Contributor guide
Research direction
Start by reading the expired-file cleanup paths for CommitLog, IndexFile, and ConsumeQueue, then reproduce the behavior with the reported 4.9.7 cluster settings while measuring IOPS. Determine which cleanup operations cause the excess I/O and document a verified way to reduce it or a focused change, with results from the reproduction as the completion criteria.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100