P2P does not play nicely with loose restrictions
Open
bug
shuffle
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 778
- Avg merge
- 2h 50m
- Merged PRs (30d)
- 3
Description
When P2P has "loose" restrictions, the scheduler will not enforce the worker restrictions P2P relies on for `unpack` tasks. As a result, P2P will be caught in a rescheduling loop until every task eventually ends up on the correct worker by chance.
Implementation notes:
We can't simply unset loose restrictions. If there are unsatisfiable restrictions, P2P will be stuck forever.
If we have loose worker restrictions, P2P should not limit itself to these workers but use the entire pool of workers.
Contributor guide
Assessment
This issue has not been assessed yet.