dask / dask/distributed

Option to turn off `reevaluate_occupancy`

Open
#4,523 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

During the call earlier, we discussed turning off `reevaluate_occupancy` and seeing how things ran. TBH I'm not actually sure how to do this. Looking at the code in a few placed, it seems like `reevaluate_occupancy` gets added to the `IOLoop` (see below), but there doesn't appear to be a clear way to influence whether it is called either by disabling or delaying the frequency of the call. Is there one and I'm just overlooking things? Or should we add one? Related we can disable stealing, which we've played with before, but this only affects things after going through a fair bit of `reevaluate_occupancy`. So not sure if this is what we had in mind. Thoughts? 🙂

https://github.com/dask/distributed/blob/383ea0326ae103b5d5e0b62ed9c3cb18510c5b9e/distributed/scheduler.py#L3273

https://github.com/dask/distributed/blob/383ea0326ae103b5d5e0b62ed9c3cb18510c5b9e/distributed/scheduler.py#L6490-L6492

cc @mrocklin @quasiben

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.