pybind / pybind/pybind11

[BUG]: pybind11 2.10.0 - error_fetch_and_normalize changes conflicts with PyPy behavior for PyErr_NormalizeException

Open
#4,075 19 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

triage
Dominant language
C++
Stars
18k
Forks
2.3k
Avg merge
5d 17h
Merged PRs (30d)
10

Description

Required prerequisites
Problem description

PyPy 3.8 triggers the newly introduced error message

https://github.com/pybind/pybind11/blob/59f03ee389c283cde65bd800c8f32ea690daf3fd/include/pybind11/pytypes.h#L459-L469

MISMATCH of original and normalized active exception types: ORIGINAL OSError REPLACED BY FileNotFoundError: [Errno 2] No such file or directory: 'this_file_does_not_exist'

In PyPy, PyErr_NormalizeException causes the "raw" OSError to be normalized into FileNotFoundError. Which seems fair enough.

It's unclear to me whether PyPy is doing something "naughty" here that violates the expectations set by CPython, or if this new exception handling code is just imposing an expectation that doesn't hold for PyPy. As such I don't know if simply disabling that test is appropriate for PyPy.

CPython does not make the same substitution. There is no error (and the code behaves correctly) with pybind11 2.9.

Reproducible example code
    m.def("test_pypy_oserror_normalization", []() {
        // We want to do "with suppress(FileNotFoundError): open(f)" in C++
        try {
            py::module_::import("io").attr("open")("this_file_does_not_exist", "r");
        } catch (const py::error_already_set &e) {
            // FileNotFoundError
        }
    });

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with include/pybind11/pytypes.h at the linked error handling around PyErr_NormalizeException, then reproduce the provided C++ example under PyPy 3.8 and CPython. Determine whether PyPy's OSError-to-FileNotFoundError normalization is expected, and document or implement compatibility behavior with regression coverage; done means the mismatch is resolved without breaking the existing CPython behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
api, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.