dask / dask/distributed

`Future.__del__` logs ignored exceptions during interpreter shutdown

Open
#8,500 1 comment 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

While I don't have a reproducer, multiple users have reported seeing many logs about `Future.__del__` ignoring exceptions during interpreter shutdown. #8449 was supposed to address this, but it failed to do so:

Before #8449

```python3
Exception ignored in:
Traceback (most recent call last):
File "/python/lib/python3.10/site-packages/distributed/client.py", line 510, in __del__
AttributeError: 'NoneType' object has no attribute 'is_finalizing'
```

After #8449
```python3
Exception ignored in:
Traceback (most recent call last):
File "/python/lib/python3.10/site-packages/distributed/client.py", line 510, in __del__
TypeError: 'NoneType' object is not callable
```

From what I understand, this fails because of this caveat: https://github.com/python/cpython/blob/e39ae6bef2c357a88e232dcab2e4b4c0f367544b/Doc/c-api/init.rst#L403-L405

Empirically, we can fix the `Future.__del__` problem by binding `sys.is_finalizing` to a class variable which will avoid its destruction. However, I assume that `is_python_shutting_down` has a similar problem. While this issue isn't urgent, the chattiness is annoying and might distract from genuine issues on shutdown.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.