SharedMemory object implement non-uniform size behaviors.
Nessuno ha ancora preso questa issue.
- Lingua principale
- Python
- Stelle
- 77.2k
- Fork
- 35.9k
- Metriche di merge delle PR
- Metriche PR in attesa
Descrizione
Bug report
Bug description:
Here's the python behavior:
The object which created SharedMemory does enforce the size specified.
As such, myblock read/writes work within the bounds of size.
>>> myblock = shared_memory.SharedMemory(create=True, name='MYBLOCK', size=8)
>>> myblock.size
8
>>> myblock.buf[7]=7
>>> myblock.buf.tobytes()
b'\x00\x00\x00\x00\x00\x00\x07'
Expectedly, errors are thrown beyond the bounds of size.
>>> myblock.buf[4095]=3
Traceback (most recent call last):
File "<python-input-37>", line 1, in <module>
myblock.buf[4095]=3
~~~~~~~~~~~^^^^^^
IndexError: index out of bounds on dimension 1
>>> myblock.buf[4094:4095]=b'k'
Traceback (most recent call last):
File "<python-input-39>", line 1, in <module>
myblock.buf[4094:4095]=b'k'
~~~~~~~~~~~^^^^^^^^^^^
ValueError: memoryview assignment: lvalue and rvalue have different structures
>>>
However, objects attaching to the same block may not have the created size bounds.
>>> from multiprocessing.shared_memory import SharedMemory
>>> attachblock = SharedMemory(name='MYBLOCK')
>>> attachblock.size
4096
>>> attachblock.buf.tobytes()
b'\x00\x00\x00\x00\x00\x00\x00\x07\x00\--excluded--'
Attached objects may read/write up to the nearest system page size without errors thrown.
>>> attachblock.buf[4095]=3
>>> attachblock.buf[4090:4091]=b'k'
>>> attachblock.buf.tobytes()
b'\x00\x00\x00\x00\x00\x00\x00\x07\--excluded--\x00k\x00\x00\x00\x00\x03'
>>>
With the stark contrast of object behavior between create=True and create=False, I would strongly argue that this behavior should be considered a bug until there is sufficient documentation describing the use case for a single named SharedMemory to both employ and present non-uniform size.
As a bugfix, I would suggest any of a few mutually exclusive solutions to provide uniform size and function.
Solution 1 [create]:
When create=True & size is specified: During instantiation, the memory size allocated by the OS is queried, and reflected in the python object's size. Subsequently, the python object allows read/writes up to the allocated size.
- Brings the implementation closer towards current documentation, needing less rework in the doc.
- Likely little work necessary on the code as it closely matches the current attachment behavior.
Solution 2 [attach]:
When create=False: allow read/writes up to only the size specified when the named SharedMemory was created.
- Python intuitive behavior i.e. you get what you expect.
- In essence, repeats
nameimplementation pattern onsize.
Solution 3: [match_system or nearest_size parameter]
Alternatively, I'd suggest a new default parameter match_system=False or nearest_size=False be created for SharedMemory which controls whether the memory size allocated by the OS is queried, and reflected in the python object's size.
Additional historical context: There is an old thread specific to documentation. https://github.com/python/cpython/issues/101623#issue-1573365102
However, in light of the demonstrated behaviors, the doc is actually not accurate nor insightful for what is going on & also should be revised to reflect the behaviors that python developers should expect when interacting with the callable. That may be a separate task from a bug.
CPython versions tested on:
3.14
Operating systems tested on:
Windows
Guida per i contributori
Apri la guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Direzione di ricerca
Inizia con multiprocessing.shared_memory.SharedMemory e con il comportamento documentato a cui si fa riferimento nell’issue 101623. Riproduci le differenze di dimensione tra create=True e create=False su Windows, quindi determina quale delle semantiche di dimensione proposte sia quella prevista prima di modificare l’implementazione o la documentazione. Il lavoro è concluso quando il comportamento è uniforme o chiaramente documentato, con una copertura di regressione per il comportamento selezionato.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Valutazione
- Stack tecnologico
- python
- Ambito
- operating-systems
- Tipo di issue
- Bug
- Difficoltà
- 5/5
- Tempo stimato
- Più di una settimana
- Stato di attività
- Ferma
- Chiarezza
- Da chiarire
- Idoneità per principianti
- 35/100