lincc-frameworks / lincc-frameworks/hyrax
Review the use of Pool in Hyrax
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 41
- Forks
- 7
- Avg merge
- 5d 1h
- Merged PRs (30d)
- 8
Description
We recently found a Pool use that caused a race condition in `InferenceDatasetWriter`. We were able to resolve this by specifying `get_context('fork')` when creating the Pool.
There are a couple other places that use multiprocessing pools, `umap` and `ChromaDBImpl` specifically. We should review those and make sure they work correctly or augment them with in the same way.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Locate the multiprocessing Pool uses in umap and ChromaDBImpl, then compare them with the InferenceDatasetWriter change that specifies get_context('fork'). Review whether those uses have the same race-condition risk and check the relevant behavior or tests. Done means the affected pool uses are verified as safe or consistently updated.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100