[Feature] Control the background job thread nums of rocksdb
- Dominant language
- Java
- Stars
- 2.1k
- Forks
- 625
- Avg merge
- 3d 14h
- Merged PRs (30d)
- 97
Description
### Search before asking
- [X] I searched in the [issues](https://github.com/alibaba/fluss/issues) and found nothing similar.
### Motivation
Currently, we will create two background threads per bucket to do fluss/compaction which may cause controllable threads.
### Solution
We can control the background threads in server level. Thanksfully, Rocksdb allows us share a threads pool across rocksdbs.
See more in https://github.com/facebook/rocksdb/blob/3ef909248736eb80e7c6b3c7a19753c4dbe9d0c5/include/rocksdb/env.h#L465
### Anything else?
_No response_
### Willingness to contribute
- [ ] I'm willing to submit a PR!
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by tracing where the server creates RocksDB instances and where the two background threads per bucket are configured. Read the linked RocksDB Env thread-pool documentation, then determine how a server-level shared pool can control those threads; done means the background thread count is configurable without creating a separate pool per bucket.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, databases
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100