python / python/cpython

Multiprocessing Semaphore and BoundedSemaphore - incomplete features ('get_value' missing) on MacOSX.

Ouverte
#125,828 2 commentaires 0 réactions 0 personnes assignées Voir sur GitHub

Personne n'a encore pris cette issue.

OS-mac stdlib topic-multiprocessing type-bug
Langage dominant
Python
Étoiles
77.2k
Forks
36k
Métriques de merge des PR
Métriques de PR en attente

Description

Bug report

Bug description:

I've been working on this PR: Adding multiprocessing queue shutdown feature for a while on my Mac machine. I've come up with a solution for this new feature and wanted to check that it also worked on Linux. However, I had forgotten that BoundedSemaphore is not fully functional on MacOSX. In fact, there's no check for exceeding the upper bound. See this example:

import multiprocessing as mp
bs = mp.BoundedSemaphore(2)
print(bs) # <BoundedSemaphore(value=unknown, maxvalue=2)>  note invalid representation string 
bs.release() # Succeed with MacOSX, Raise an exception on Linux 

I had to revise my solution because it was invalid on Linux. Such a shame, I wasted a lot of time. I checked on the documentation a few days ago . There is not a specific warning about this case.

As a reminder, the initial reason is the absence of the sem_getvalue C function in the MacOSX semaphore implementation. This prevents retrieval of the current semaphore value. It also prevents the Semaphore use as a shared counter. This issue is (very ?) old, and still hasn't been solved from Apple, could a workaround be implemented on MacOSX, so that Semaphore and BoundedSemaphore would have the same functionalities as Linux/Win.

A workaround (in Python) would be to use a shared value and to revise the acquire and release methods in order to emulate the semaphore internal counter. It is quite possible that this solution would greatly reduce performance.

I've already done a lot of tests (See the attached draft PR). And also, this solution should fix the representation strings that are currently wrong.

Before going any further, I was wondering if there's a reason why no patch attempt has been implemented ? And was there an old discussion about this already ?

Thanks for your feedback

CPython versions tested on:

3.10, 3.11, 3.12, 3.13, CPython main branch

Operating systems tested on:

macOS

Linked PRs
  • gh-125829
  • gh-130913

Guide de contribution

Ouvrir le guide de contribution

Par où commencer

  1. Lisez l'issue en entier, puis le guide de contribution du projet.
  2. Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
  3. Forkez le dépôt et travaillez sur une branche.
  4. Ouvrez une pull request qui référence le numéro de l'issue.

Piste de recherche

Commencez par multiprocessing.Semaphore et BoundedSemaphore sur macOS, en vous concentrant sur le comportement manquant de sem_getvalue et sur les chemins acquire/release. Examinez la PR brouillon liée et comparez la représentation présentée ainsi que le comportement de vérification des limites avec Linux ; la finalisation nécessiterait un contournement convenu et une couverture des valeurs, des limites et de la représentation.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Évaluation

Stack technique
python
Domaine
operating-systems
Type d'issue
Bug
Difficulté
5/5
Temps estimé
Plus d'une semaine
Activité
À l'abandon
Clarté
À clarifier
Accessibilité débutants
20/100

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.