python / python/cpython

asyncio.StreamReader raises BufferError with active memoryview

未關閉
#146,379 9 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視

還沒有人認領這個 Issue。

stdlib topic-asyncio type-bug
主要語言
Python
星號
77.2k
分支
36k
PR 合併指標
PR 指標待擷取

描述

Bug report

Bug description:
Summary

When StreamReader has an active memoryview export of its internal buffer, methods that mutate the buffer in place can raise:

BufferError: Existing exports of data: object cannot be re-sized

This appears in real workloads with async database drivers, but the root cause is in StreamReader buffer mutation behavior.

Steps to reproduce
import asyncio

async def main():
    r = asyncio.StreamReader()
    r.feed_data(b"AABBCCDD")

    # Simulate a consumer that keeps a memoryview alive
    mv = memoryview(r._buffer)

    # Triggers BufferError in current implementation
    await r.readexactly(4)

    mv.release()

asyncio.run(main())
Expected behavior

readexactly(4) should return b"AABB" without raising BufferError.

Actual behavior

BufferError: Existing exports of data: object cannot be re-sized

Why this happens

Some StreamReader paths mutate bytearray in place, for example slice deletion and clear operations. In-place resize is not allowed while a memoryview export exists.

Affected methods
  • read
  • readline
  • readuntil
  • readexactly
Environment
  • Python: 3.13.x, 3.14.x (also reproducible in local tests with newer builds)
  • OS: macOS (also observed in Linux server workloads)
  • Third-party context where this is frequently hit: async DB drivers using memoryview over packet buffers
Possible fix direction

Replace in-place buffer mutations with non-mutating replacement assignments, for example:

  • replace slice delete with reassignment to remaining bytes
  • replace clear with assigning a new empty bytearray

If useful, I can open a PR with tests covering the regression paths.

CPython versions tested on:

3.13

Operating systems tested on:

macOS

Linked PRs
  • gh-146380

貢獻指南

開啟貢獻指南

從這裡開始

  1. 先讀完整個 Issue,再讀專案的貢獻指南。
  2. 在 Issue 下留言說明你要接手 —— 這能避免兩個人做同樣的事。
  3. Fork 儲存庫,在一個分支上完成修改。
  4. 送出 Pull Request,並在描述裡引用這個 Issue 編號。

研究方向

從 asyncio.StreamReader 以及報告中描述的 read、readline、readuntil 和 readexactly 路徑開始;執行提供的重現步驟,以觀察存在作用中的 memoryview 時的 BufferError。完成標準是這些方法在不引發例外的情況下回傳預期資料,並為受影響的緩衝區變更路徑提供回歸涵蓋率。

由索引模型根據 Issue 內容生成。

評估

技術堆疊
python
領域
backend
Issue 類型
缺陷
難度
3/5
預估耗時
1-2 天
活躍度
停滯
描述清晰度
基本清楚
新手友好度
35/100

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。