python / python/cpython

SharedMemory object implement non-uniform size behaviors.

Ouverte
#145,954 6 commentaires 0 réactions 0 personnes assignées Voir sur GitHub

Personne n'a encore pris cette issue.

stdlib topic-multiprocessing type-feature
Langage dominant
Python
Étoiles
77.2k
Forks
35.9k
Métriques de merge des PR
Métriques de PR en attente

Description

Bug report

Bug description:

Here's the python behavior:

The object which created SharedMemory does enforce the size specified.
As such, myblock read/writes work within the bounds of size.

>>> myblock = shared_memory.SharedMemory(create=True, name='MYBLOCK', size=8)
>>> myblock.size
8
>>> myblock.buf[7]=7
>>> myblock.buf.tobytes()
b'\x00\x00\x00\x00\x00\x00\x07'

Expectedly, errors are thrown beyond the bounds of size.

>>> myblock.buf[4095]=3
Traceback (most recent call last):
  File "<python-input-37>", line 1, in <module>
    myblock.buf[4095]=3
    ~~~~~~~~~~~^^^^^^
IndexError: index out of bounds on dimension 1
>>> myblock.buf[4094:4095]=b'k'
Traceback (most recent call last):
  File "<python-input-39>", line 1, in <module>
    myblock.buf[4094:4095]=b'k'
    ~~~~~~~~~~~^^^^^^^^^^^
ValueError: memoryview assignment: lvalue and rvalue have different structures
>>>

However, objects attaching to the same block may not have the created size bounds.

>>> from multiprocessing.shared_memory import SharedMemory
>>> attachblock = SharedMemory(name='MYBLOCK')
>>> attachblock.size
4096
>>> attachblock.buf.tobytes()
b'\x00\x00\x00\x00\x00\x00\x00\x07\x00\--excluded--'

Attached objects may read/write up to the nearest system page size without errors thrown.

>>> attachblock.buf[4095]=3
>>> attachblock.buf[4090:4091]=b'k'
>>> attachblock.buf.tobytes()
b'\x00\x00\x00\x00\x00\x00\x00\x07\--excluded--\x00k\x00\x00\x00\x00\x03'
>>>

With the stark contrast of object behavior between create=True and create=False, I would strongly argue that this behavior should be considered a bug until there is sufficient documentation describing the use case for a single named SharedMemory to both employ and present non-uniform size.

As a bugfix, I would suggest any of a few mutually exclusive solutions to provide uniform size and function.

Solution 1 [create]:

When create=True & size is specified: During instantiation, the memory size allocated by the OS is queried, and reflected in the python object's size. Subsequently, the python object allows read/writes up to the allocated size.

  • Brings the implementation closer towards current documentation, needing less rework in the doc.
  • Likely little work necessary on the code as it closely matches the current attachment behavior.
Solution 2 [attach]:

When create=False: allow read/writes up to only the size specified when the named SharedMemory was created.

  • Python intuitive behavior i.e. you get what you expect.
  • In essence, repeats name implementation pattern on size.
Solution 3: [match_system or nearest_size parameter]

Alternatively, I'd suggest a new default parameter match_system=False or nearest_size=False be created for SharedMemory which controls whether the memory size allocated by the OS is queried, and reflected in the python object's size.


Additional historical context: There is an old thread specific to documentation. https://github.com/python/cpython/issues/101623#issue-1573365102
However, in light of the demonstrated behaviors, the doc is actually not accurate nor insightful for what is going on & also should be revised to reflect the behaviors that python developers should expect when interacting with the callable. That may be a separate task from a bug.

CPython versions tested on:

3.14

Operating systems tested on:

Windows

Guide de contribution

Ouvrir le guide de contribution

Par où commencer

  1. Lisez l'issue en entier, puis le guide de contribution du projet.
  2. Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
  3. Forkez le dépôt et travaillez sur une branche.
  4. Ouvrez une pull request qui référence le numéro de l'issue.

Piste de recherche

Commencez par multiprocessing.shared_memory.SharedMemory et le comportement documenté référencé dans l’issue 101623. Reproduisez les différences de taille entre create=True et create=False sous Windows, puis déterminez laquelle des sémantiques de taille proposées est souhaitée avant de modifier l’implémentation ou la documentation. Le travail est terminé lorsque le comportement est uniforme ou clairement documenté, avec une couverture de régression pour le comportement sélectionné.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Évaluation

Stack technique
python
Domaine
operating-systems
Type d'issue
Bug
Difficulté
5/5
Temps estimé
Plus d'une semaine
Activité
À l'abandon
Clarté
À clarifier
Accessibilité débutants
35/100

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.