dask / dask/distributed

`send_task_state_to_scheduler` may not need `self.data[key]`

Open
#3,629 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

Currently in [`Worker.send_task_state_to_scheduler(...)`]( https://github.com/dask/distributed/blob/f2f82c6c2e8d36731cb3fb82fb1f80ea0323358e/distributed/worker.py#L1837 ), we check [`self.data[key]` or `self.actors[key]`]( https://github.com/dask/distributed/blob/f2f82c6c2e8d36731cb3fb82fb1f80ea0323358e/distributed/worker.py#L1839-L1842 ) and then determine relevant metadata about that key like its [`nbytes` and `typ`]( https://github.com/dask/distributed/blob/f2f82c6c2e8d36731cb3fb82fb1f80ea0323358e/distributed/worker.py#L1843-L1844 ). While the code appears to handle [access of `self.nbytes` with a fallback to using `value`]( https://github.com/dask/distributed/blob/f2f82c6c2e8d36731cb3fb82fb1f80ea0323358e/distributed/worker.py#L1843 ), the access pattern using `__getitem__`, which would raise a `KeyError` if the `key` wasn't there. This makes one wonder if that fallback to get `value` is even needed here.

For context, this came up in this discussion ( https://github.com/dask/distributed/pull/3628#pullrequestreview-379910338 ).

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.