`pytest`'s`LogCaptureHandler` prevents garbage collection of `Worker` instances (and others)
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 778
- Avg merge
- 2h 50m
- Merged PRs (30d)
- 3
Description
In tests that raise an exception, the `LogCaptureHandler` from `pytest` prevents garbage collection by maintaining a reference to the raised exception and its traceback:

_(This reference chain was created with `objgraph.find_backref_chain` [https://mg.pov.lt/objgraph/#memory-leak-example](https://mg.pov.lt/objgraph/#memory-leak-example))_
Keeping those references alive prevents garbage collection of otherwise released objects and therefore testing for proper garbage collection of `Scheduler`, `TaskState` and `Worker` instances as requested by #6250. For example, in #6344, the failures of `distributed/tests/test_scheduler.py::test_rebalance_workers_and_keys` and several other tests are caused by this.
One possible for the failures in #6344 would be to manually clear the handlers during `check_instances()`, though they seem to be hard to access, which results in a rather brittle solution:
```
handlers = logging.getLogger().handlers
for handler in handlers:
if hasattr(handler, "records"):
handler.records.clear()
```
Contributor guide
Assessment
This issue has not been assessed yet.