SpikeInterface / SpikeInterface/spikeinterface
Pick better default for mp_context
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 847
- Forks
- 280
- Avg merge
- 3d 9h
- Merged PRs (30d)
- 29
Description
#4333 changes how mp_context is chosen from:
to
Before the PR; job_kwargs.mp_context=None and recording.get_preferred_mp_context()=None (the default and most common setup) will call mp_context = multiprocessing.get_context(None) that returned the default context for that specific machine (spawn for windows & Mac and fork/forkserver for Linux). Now, the behavior has changed for Linux: it always picks fork. Default mp_context for Linux from python 3.14 onwards is forkserver (a thread-safe, less problematic alternative to fork) docs.
When running kilosort4 in python 3.14. fork did not play nice with some multi-threaded calls inside scikit learn, particularly the process hangs forever in this line:
https://github.com/MouseLand/Kilosort/blob/7a19a57bef39f3e07ab7fa1edf5aa8635c69a850/kilosort/spikedetect.py#L72
This was run inside a kubernetes pod with GPUs but I couldn't replicate it locally: 3.13 and 3.14 both worked with fork or forkserver :/ so not sure if it will be a problem for other people.
In any case, it might be worth considering making forkserver (rather than fork) the Linux default (it's been around since python 3.4) or allow multiprocessing to pick the best default based on system and python version as it did before this PR, i.e., let mp_context=None with a comment saying what that means).
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in src/spikeinterface/core/job_tools.py around lines 418-440 and compare the earlier selection at lines 419-427. Review Python's multiprocessing context defaults and the reported Kilosort/scikit-learn hang, especially the referenced spikedetect.py line 72. Done means agreeing on and implementing a safe default or preserving None with clear documentation, then validating the affected workload.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100