dask / dask/distributed

Missing test: duplicates in Worker.ready

Open
#6,096 0 comments 0 reactions 0 assignees View on GitHub
tests
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

Follow-up from #6062

1. A task is assigned to a Worker
2. The task transitions to ready, thus is pushed into the ``Worker.ready`` heap, or to constrained, thus is pushed into the ``Worker.constrained`` heap
3. Before it reaches computing state, it is stolen. As there's no way to pop from the middle of a heap, the task stays there.
4. The task is assigned to the worker again before ``Worker._ensure_computing`` could reach its position in the heap
5. You end up with a duplicate key in the heap.

There's nothing wrong with having a duplicate _per se_, and ``Worker._ensure_computing`` gracefully deals with it; however the use case is far from trivial. At the moment of writing, replacing the graceful handling with a hard assertion of no duplicates only causes very mild flakiness in the most demanding stress tests.

Write a unit test that deterministically triggers the use case both with and without resource constraints.

CC @fjetter

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.