Delegate cost computation in CostBalancerStrategy to the common ForkJoinPool and remove balancerComputeThreads config
- Dominant language
- Java
- Stars
- 14.1k
- Forks
- 3.8k
- Avg merge
- 2d 58m
- Merged PRs (30d)
- 233
Description
I don't see a point of using custom thread pool for `CostBalancerStrategy`, because it delegates to the pool "pure", non-blocking `computeCost()` calls. When the logic to be parallelized is non-blocking, it's a job for the common `ForkJoinPool` via parallel Streams.
This will likely also improve scalability because instead of wrapping each `computeCost()` call as a separate task for a thread pool (this might be the reason for poor scalability - see #3275), parallel `Stream` won't execute more slices than there are cores in the system.
In the transition period, the `balancerComputeThreads` may be ignored and a warning or an alert is issued if this option is found in the config.
FYI @himadrisingh001 @xvrl @fjy - contributors to the original change that introduced parallelization (#399).
Contributor guide
Research direction
Start by reading CostBalancerStrategy and its computeCost() parallelization, then trace where balancerComputeThreads is read and applied. Replace the custom pool usage with the common ForkJoinPool through parallel Streams, and verify that the legacy configuration is ignored with a warning or alert when present.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, performance
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100