scverse / scverse/fast-array-utils
More `jit` options, specifically `nogil=True` on the serial kernels
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 15
- Forks
- 5
- Avg merge
- 10h 33m
- Merged PRs (30d)
- 9
Description
Hi @flying-sheep I was wondering why the serial (fallback) kernels aren't compiled with nogil=True. This would enable parallelization via joblib and dask schedulers. Is there a reason for it? If not I can open a PR showing the speedup also.
I also want to point to something I learned. Tbb doesn't oversubscribe because it uses a process-wide arena. So, we can suggest tbb in certain OS'es instead of falling back. But setting nogil=True will be enough to be fast enough in itself.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the serial fallback kernels and the current jit configuration in the repository. Check how their compilation options are set and whether nogil=True can be applied consistently. Done means the serial kernels support the requested option without breaking their existing behavior, with the claimed parallelization speedup demonstrated as described in the issue.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- performance
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 65/100