Missing test: duplicates in Worker.ready
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 778
- Avg merge
- 2h 50m
- Merged PRs (30d)
- 3
Description
Follow-up from #6062
1. A task is assigned to a Worker
2. The task transitions to ready, thus is pushed into the ``Worker.ready`` heap, or to constrained, thus is pushed into the ``Worker.constrained`` heap
3. Before it reaches computing state, it is stolen. As there's no way to pop from the middle of a heap, the task stays there.
4. The task is assigned to the worker again before ``Worker._ensure_computing`` could reach its position in the heap
5. You end up with a duplicate key in the heap.
There's nothing wrong with having a duplicate _per se_, and ``Worker._ensure_computing`` gracefully deals with it; however the use case is far from trivial. At the moment of writing, replacing the graceful handling with a hard assertion of no duplicates only causes very mild flakiness in the most demanding stress tests.
Write a unit test that deterministically triggers the use case both with and without resource constraints.
CC @fjetter
Contributor guide
Assessment
This issue has not been assessed yet.