dask / dask/distributed

Ensure TaskState instances are released on Scheduler and Worker

Open
#6,250 2 comments 0 reactions 1 assignee Claimed by @fjetter View on GitHub
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

There are strong references on our TaskState instances, Scheduler and Worker instances

See for instance https://github.com/dask/distributed/pull/6246

We should ensure that TaskState is are properly cleaned up

- There is a test in Scheduler.validate that ensure TaskState instances are properly released https://github.com/dask/distributed/pull/6226#issuecomment-1113086051
- `utils_test.check_instances` verifies that there are no leftover Scheduler, Worker, Nanny or TaskState instances

Caveat: This depends on the profiler thread so we may need to relax the above tests and replace them with a more robust, less strict version as appropriate

https://github.com/dask/distributed/pull/6246

- [ ] https://github.com/dask/distributed/pull/6336
- [ ] https://github.com/dask/distributed/pull/6344
- [ ] https://github.com/dask/distributed/pull/6345

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.