python / python/cpython

Use-after-free in _Unpickler_ReadIntoFromFile: temporary memoryview passed to readinto() can outlive its buffer

未关闭
#151,046 1 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

还没有人认领这个 Issue。

extension-modules type-crash
主要语言
Python
星标
77.2k
派生
35.9k
PR 合并指标
PR 指标待抓取

描述

Bug report

Bug description

The C implementation of pickle.Unpickler, when reading from a file-like
object that provides readinto(), hands that method a temporary memoryview
created over an internal buffer:

PyObject *buf_obj = PyMemoryView_FromMemory(buf, n, PyBUF_WRITE);
...
PyObject *read_size_obj = _Pickle_FastCall(self->readinto, buf_obj);

(Modules/_pickle.c, _Unpickler_ReadIntoFromFile.)

buf points into a short-lived buffer (e.g. the bytes object allocated in
load_counted_binbytes, which may also be reallocated by _PyBytes_Resize or
freed when unpickling ends). The memoryview is never released or invalidated
after readinto() returns. A readinto() implementation that keeps a
reference to the view can therefore use it to read or write the buffer after
it has been freed, which is a use-after-free at the C level.

This only requires a pure-Python file-like object -- no ctypes. A pure-Python
program should not be able to make the interpreter read or write freed memory.

Reproducer

import pickle, struct, gc

stashed = []

class EvilFile:
    def __init__(self):
        self._h = b"\x80\x05" + b"\x8e" + struct.pack("<Q", 200_000)
        self._p = 0
    def read(self, n=-1):
        d = self._h[self._p:] if (n is None or n < 0) else self._h[self._p:self._p+n]
        self._p += len(d); return d
    def readline(self):
        return self.read(-1)
    def readinto(self, view):
        stashed.append(view)             # keep the view past readinto()
        view[:] = b"A" * len(view); return len(view)

up = pickle.Unpickler(EvilFile())
try:
    up.load()                            # stream ends after the payload
except EOFError:
    pass
del up; gc.collect()                     # free the backing buffer
_ = [bytes(200_000) for _ in range(8)]   # churn the allocator
stashed[0][0]                            # <-- use-after-free read

On a --with-address-sanitizer --with-pydebug build this reports a clean
heap-use-after-free (READ in unpack_single, the buffer freed via
Pdata_dealloc and originally allocated in load_counted_binbytes).

Root cause and fix direction

The view is a non-owning window over a raw pointer that the unpickler does not
keep alive. Other CPython sites that drive a user readinto() (e.g.
_io.RawIOBase.read()) hand out an owning object (a bytearray) whose buffer
protocol prevents it from being freed while still exported. The pickle path
should release the temporary memoryview as soon as readinto() returns, so a
surviving reference raises ValueError: operation forbidden on released memoryview object instead of dereferencing freed memory.

I have a patch with a regression test and will open a PR.

(For context: this was originally raised privately with the Python Security
Response Team, who advised opening a public issue.)

CPython versions tested on

3.16.0a0 (main, commit 5755d0f).

Operating systems tested on

macOS (arm64), --with-address-sanitizer --with-pydebug build.

Linked PRs
  • gh-151048

贡献指南

打开贡献指南

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

调研方向

从 Modules/_pickle.c 中的 _Unpickler_ReadIntoFromFile 开始,检查 readinto() 返回后临时 memoryview 的处理方式。在 ASAN、pydebug 构建中使用提供的 EvilFile 复现程序,然后添加或检查提到的回归测试。当保留的视图无法访问已释放的缓冲区并引发预期的已释放 memoryview 错误时,即表示完成。

由索引模型根据 Issue 内容生成。

评估

技术栈
c, python
领域
backend, security
Issue 类型
缺陷
难度
4/5
预计耗时
3-5 天
活跃度
停滞
描述清晰度
描述清楚
新手友好度
20/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。