MemoryMonitor thrashing detection is too aggressive for batch workers
Open
dataflow
improvement
P3
runners
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
In the streaming dataflow worker we've implemented push-back so that we will reduce parallelism when there is memory pressure. Since we cannot do this on the batch worker, it doesn't make sense for our thrashing detection to be so aggressive. We should increase the thresholds used for thrashing detection when running on Batch workloads.
Imported from Jira [BEAM-9049](https://issues.apache.org/jira/browse/BEAM-9049). Original Jira may contain additional context.
Reported by: bhulette.
Contributor guide
Assessment
This issue has not been assessed yet.