`DequeHandler` prevents garbage collection of `Worker` instances and others
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 778
- Avg merge
- 2h 50m
- Merged PRs (30d)
- 3
Description
In tests that raise an exception, the `DequeHandler` prevents garbage collection by maintaining a reference to the raised exception and its traceback:

_(This reference chain was created with `objgraph.find_backref_chain` [https://mg.pov.lt/objgraph/#memory-leak-example](https://mg.pov.lt/objgraph/#memory-leak-example))_
Keeping those references alive prevents garbage collection of otherwise released objects and therefore testing for proper garbage collection of `Scheduler`, `TaskState` and `Worker` instances as requested by #6250. For example, in #6344, the failures of `distributed/tests/test_worker.py::test_heartbeat_comm_closed` and several other tests are caused by this.
One possible solution to enable garbage collection in tests would be to call `DequeHandler.clear_all_instances()` in `check_instances()` before checking for proper garbage collection, though there might be superior options that generally improve our logging.
Contributor guide
Assessment
This issue has not been assessed yet.