3.11: _PyUnicode_Equal call sites lack error handling

还没有人认领这个 Issue。

(Found while looking at https://github.com/python/cpython/issues/98783)

In https://github.com/python/cpython/commit/81c72044a181dbbfbf689d7a977d0d99090f26a8, several uses of _PyUnicode_EqualToASCIIId, which cannot fail, were replaced with _PyUnicode_Equal, which can fail: it calls PyUnicode_READY() in Python 3.11, and returns -1 on failure.

In Python 3.12, this is not a problem: _PyUnicode_Equal cannot fail, since it does not need to call PyUnicode_READY() since the wstr APIs were removed in https://github.com/python/cpython/commit/f9c9354a7a173eaca2aa19e667b5cf12167b7fed. But it is theoretically a problem in 3.11.

In 3.11, git grep "_PyUnicode_Equal(" turns up the following:

Include/cpython/unicodeobject.h:PyAPI_FUNC(int) _PyUnicode_Equal(PyObject *, PyObject *);
Modules/_pickle.c:            use_newobj_ex = _PyUnicode_Equal(name, &_Py_ID(__newobj_ex__));
Modules/_pickle.c:                use_newobj = _PyUnicode_Equal(name, &_Py_ID(__newobj__));
Objects/longobject.c:    else if (_PyUnicode_Equal(byteorder, &_Py_ID(little)))
Objects/longobject.c:    else if (_PyUnicode_Equal(byteorder, &_Py_ID(big)))
Objects/longobject.c:    else if (_PyUnicode_Equal(byteorder, &_Py_ID(little)))
Objects/longobject.c:    else if (_PyUnicode_Equal(byteorder, &_Py_ID(big)))
Objects/typeobject.c:    if (mod != NULL && !_PyUnicode_Equal(mod, &_Py_ID(builtins)))
Objects/typeobject.c:        if (_PyUnicode_Equal(name, &_Py_ID(__dict__))) {
Objects/typeobject.c:        if (_PyUnicode_Equal(name, &_Py_ID(__weakref__))) {
Objects/typeobject.c:        if ((ctx->add_dict && _PyUnicode_Equal(slot, &_Py_ID(__dict__))) ||
Objects/typeobject.c:            (ctx->add_weak && _PyUnicode_Equal(slot, &_Py_ID(__weakref__))))
Objects/typeobject.c:            if (!_PyUnicode_Equal(slot, &_Py_ID(__qualname__)) &&
Objects/typeobject.c:                !_PyUnicode_Equal(slot, &_Py_ID(__classcell__)))
Objects/typeobject.c:    if (mod != NULL && !_PyUnicode_Equal(mod, &_Py_ID(builtins)))
Objects/typeobject.c:        _PyUnicode_Equal(name, &_Py_ID(__class__)))
Objects/typeobject.c:        if (_PyUnicode_Equal(name, &_Py_ID(__class__))) {
Objects/unicodeobject.c:_PyUnicode_Equal(PyObject *str1, PyObject *str2)
Python/ceval.c:            int res = _PyUnicode_Equal(left, right);
Python/errors.c:        if (!_PyUnicode_Equal(modulename, &_Py_ID(builtins)) &&
Python/errors.c:            !_PyUnicode_Equal(modulename, &_Py_ID(__main__))) {
Python/pythonrun.c:        if (!_PyUnicode_Equal(modulename, &_Py_ID(builtins)) &&
Python/pythonrun.c:            !_PyUnicode_Equal(modulename, &_Py_ID(__main__)))

Broken down:

_pickle.c
- Could the result of _PyObject_LookupAttr(callable, &_Py_ID(__name__), &name) be unready?
longobject.c
- Safe because argument clinic calls PyUnicode_READY()
typeobject.c
- ctx->slots could be an arbitrary tuple of potentially-unready strings, so there should be some call to PyUnicode_READY() at or before type_new_visit_slots.
ceval.c
- Already has the error checking (though it could be removed in 3.12!)
errors.c and pythonrun.c:
- Arbitrary result of PyObject_GetAttr(exc_type, &_Py_ID(__module__)); might not be _READY()?

The good news is that this would be hard to run into in practice: it requires both using the old deprecated wstr APIs and running into a memory error during PyUnicode_READY().

cc @ericsnowcurrently @methane

先在 Python 3.11 分支上使用 git grep 查找 _PyUnicode_Equal，然后阅读 Modules/_pickle.c、Objects/longobject.c、Objects/typeobject.c、Python/ceval.c、Python/errors.c 和 Python/pythonrun.c 中列出的调用点。跟踪每个输入是否可能是尚未准备好的 Unicode 对象，以及 PyUnicode_READY() 的失败如何传播。受影响的调用点具有安全的错误处理或经过验证的就绪保证后，即表示完成。

由索引模型根据 Issue 内容生成。

描述

贡献指南

从这里开始

调研方向

评估

3.11: _PyUnicode_Equal call sites lack error handling

描述

贡献指南

从这里开始

调研方向

评估

把新 issue 发到你的邮箱