python / python/cpython

SharedMemory object implement non-uniform size behaviors.

未關閉
#145,954 6 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視

還沒有人認領這個 Issue。

stdlib topic-multiprocessing type-feature
主要語言
Python
星號
77.2k
分支
36k
PR 合併指標
PR 指標待擷取

描述

Bug report

Bug description:

Here's the python behavior:

The object which created SharedMemory does enforce the size specified.
As such, myblock read/writes work within the bounds of size.

>>> myblock = shared_memory.SharedMemory(create=True, name='MYBLOCK', size=8)
>>> myblock.size
8
>>> myblock.buf[7]=7
>>> myblock.buf.tobytes()
b'\x00\x00\x00\x00\x00\x00\x07'

Expectedly, errors are thrown beyond the bounds of size.

>>> myblock.buf[4095]=3
Traceback (most recent call last):
  File "<python-input-37>", line 1, in <module>
    myblock.buf[4095]=3
    ~~~~~~~~~~~^^^^^^
IndexError: index out of bounds on dimension 1
>>> myblock.buf[4094:4095]=b'k'
Traceback (most recent call last):
  File "<python-input-39>", line 1, in <module>
    myblock.buf[4094:4095]=b'k'
    ~~~~~~~~~~~^^^^^^^^^^^
ValueError: memoryview assignment: lvalue and rvalue have different structures
>>>

However, objects attaching to the same block may not have the created size bounds.

>>> from multiprocessing.shared_memory import SharedMemory
>>> attachblock = SharedMemory(name='MYBLOCK')
>>> attachblock.size
4096
>>> attachblock.buf.tobytes()
b'\x00\x00\x00\x00\x00\x00\x00\x07\x00\--excluded--'

Attached objects may read/write up to the nearest system page size without errors thrown.

>>> attachblock.buf[4095]=3
>>> attachblock.buf[4090:4091]=b'k'
>>> attachblock.buf.tobytes()
b'\x00\x00\x00\x00\x00\x00\x00\x07\--excluded--\x00k\x00\x00\x00\x00\x03'
>>>

With the stark contrast of object behavior between create=True and create=False, I would strongly argue that this behavior should be considered a bug until there is sufficient documentation describing the use case for a single named SharedMemory to both employ and present non-uniform size.

As a bugfix, I would suggest any of a few mutually exclusive solutions to provide uniform size and function.

Solution 1 [create]:

When create=True & size is specified: During instantiation, the memory size allocated by the OS is queried, and reflected in the python object's size. Subsequently, the python object allows read/writes up to the allocated size.

  • Brings the implementation closer towards current documentation, needing less rework in the doc.
  • Likely little work necessary on the code as it closely matches the current attachment behavior.
Solution 2 [attach]:

When create=False: allow read/writes up to only the size specified when the named SharedMemory was created.

  • Python intuitive behavior i.e. you get what you expect.
  • In essence, repeats name implementation pattern on size.
Solution 3: [match_system or nearest_size parameter]

Alternatively, I'd suggest a new default parameter match_system=False or nearest_size=False be created for SharedMemory which controls whether the memory size allocated by the OS is queried, and reflected in the python object's size.


Additional historical context: There is an old thread specific to documentation. https://github.com/python/cpython/issues/101623#issue-1573365102
However, in light of the demonstrated behaviors, the doc is actually not accurate nor insightful for what is going on & also should be revised to reflect the behaviors that python developers should expect when interacting with the callable. That may be a separate task from a bug.

CPython versions tested on:

3.14

Operating systems tested on:

Windows

貢獻指南

開啟貢獻指南

從這裡開始

  1. 先讀完整個 Issue,再讀專案的貢獻指南。
  2. 在 Issue 下留言說明你要接手 —— 這能避免兩個人做同樣的事。
  3. Fork 儲存庫,在一個分支上完成修改。
  4. 送出 Pull Request,並在描述裡引用這個 Issue 編號。

研究方向

從 multiprocessing.shared_memory.SharedMemory 以及 issue 101623 中引用的文件化行為開始。在 Windows 上重現 create=True 和 create=False 的大小差異,然後在變更實作或文件之前,確定所提議的大小語義中哪一種是預期的。完成表示行為是一致的或有清楚的文件說明,並且為所選行為提供回歸測試涵蓋。

由索引模型根據 Issue 內容生成。

評估

技術堆疊
python
領域
operating-systems
Issue 類型
缺陷
難度
5/5
預估耗時
一週以上
活躍度
停滯
描述清晰度
需要釐清
新手友好度
35/100

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。