dask / dask/distributed

`DequeHandler` prevents garbage collection of `Worker` instances and others

Open
#6,362 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

In tests that raise an exception, the `DequeHandler` prevents garbage collection by maintaining a reference to the raised exception and its traceback:
![worker_chain](https://user-images.githubusercontent.com/2699097/169005149-2d2af369-9add-42f1-b274-bf9df33739cf.png)
_(This reference chain was created with `objgraph.find_backref_chain` [https://mg.pov.lt/objgraph/#memory-leak-example](https://mg.pov.lt/objgraph/#memory-leak-example))_

Keeping those references alive prevents garbage collection of otherwise released objects and therefore testing for proper garbage collection of `Scheduler`, `TaskState` and `Worker` instances as requested by #6250. For example, in #6344, the failures of `distributed/tests/test_worker.py::test_heartbeat_comm_closed` and several other tests are caused by this.

One possible solution to enable garbage collection in tests would be to call `DequeHandler.clear_all_instances()` in `check_instances()` before checking for proper garbage collection, though there might be superior options that generally improve our logging.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.