pybind / pybind/pybind11

Add noexcept(false) to destructors for gil_scoped_release

Open
#2,215 12 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
18k
Forks
2.3k
Avg merge
5d 17h
Merged PRs (30d)
10

Description

When a thread attempts to acquire the GIL but the Python interpreter has already destructed, Python will attempt to terminate the thread using pthread_exit. In many implementations of pthread_exit, this will trigger a stack unwinding, which will immediately call std::terminate if you are inside a destructor with noexcept(true). Which is the case for the destructor of gil_scoped_release, which will attempt to acquire the GIL on reentry. The net effect of this any code which uses gil_scoped_release and is called from a daemon thread from Python is likely to cause your process to unceremoniously exit.

The fix seems to be quite simple:

diff --git a/include/pybind11/pybind11.h b/include/pybind11/pybind11.h
index 7fa0f0e..eb501a7 100644
--- a/include/pybind11/pybind11.h
+++ b/include/pybind11/pybind11.h
@@ -1954,7 +1954,7 @@ public:
             PYBIND11_TLS_DELETE_VALUE(key);
         }
     }
-    ~gil_scoped_release() {
+    ~gil_scoped_release() noexcept(false) {
         if (!tstate)
             return;
         PyEval_RestoreThread(tstate);

Do you agree with this change? If so I can submit a PR for it. More context: https://github.com/pytorch/pytorch/issues/38228

It would be reasonably simple to produce a repro that doesn't involve PyTorch, please let me know if that would be helpful.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in include/pybind11/pybind11.h at the gil_scoped_release destructor and review the issue's proposed exception specification. Check the surrounding GIL and thread-shutdown behavior, then validate the change with a focused reproduction involving interpreter destruction and a daemon thread. Done means the failure no longer terminates the process unexpectedly.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
backend-api-design
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.