Creating a sparse analyzer with multiprocessing crashes if sorting has a lot of units

When creating a new analyzer from a sorting object with a few hundred units, it can crash if n_jobs is set to use more than one core.

From a recording with ~1500 units :

import spikeinterface.full as si
from pathlib import Path

path = r'/path/to/folder/sc2_output'
recording = si.load(Path(path).joinpath('spikeinterface_recording.json'))
sorting = si.load(Path(path).joinpath('sorter_output/sorting'))
analyzer = si.create_sorting_analyzer(recording=recording,
                                        sorting=sorting,
                                        sparse=True,
                                        format='memory',
                                        n_jobs=0.25,
                                      )

Results in the following :

/home/user/.conda/envs/spike/lib/python3.12/multiprocessing/resource_tracker.py:254: UserWarning: resource_tracker: There appear to be 1 leaked shared_memory objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '

Or a stuck kernel in a notebook.

This seems to be dependant on both the number of cores and the number of units. By selecting n units with :

sorting = sorting.select_units(sorting.unit_ids[0:n])

I can see the limit is about 800 units with n_jobs=0.25, which on my machine is 10 cores.
If I lower n_jobs to 0.15, which is 6 cores, I can go up to 1000 units, and 4 cores can handle the full sorting.

This is the case when creating both in-memory and on-disk analyzers (binary folder and zarr). I've monitored ressources during the process and it does not look like an OOM error.

This can get pretty constraining specifically because of estimate_sparsity, which can take a while on long recordings with no parallel processing. Afterwards, computing extensions on the full recording with 10 cores doesn't cause any problem.

Start at the create_sorting_analyzer entry point and reproduce the failure with the provided recording, sorting, sparse=True, and varying n_jobs and unit counts. Compare in-memory, binary-folder, and zarr analyzers, then inspect estimate_sparsity and multiprocessing resource handling. Done means large sparse analyzers complete without a leaked shared-memory warning or a stuck kernel.