SpikeInterface / SpikeInterface/spikeinterface
Possible issue in whitening procedure?
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 847
- Forks
- 280
- Avg merge
- 3d 9h
- Merged PRs (30d)
- 29
Description
Hello everyone,
I am currently running spikeinterface 0.101.2 to process some MaxTwo data. I concatenated some recording segments and centered the data, as I assumed it might be relevant to the issue but it doesn't change the outcome. In the process of running SpykingCircus2 I get an error with the attached error trace basically telling me that the whitening procedure of sklearn has a type mismatch. I have checked that my data is properly conditioned (no nan values, non-singular covariance matrix). I have also found out that if I step in and use the debugging console to change the dtype of the data array to float64 (i.e. the requested double type) it works fine. I have also tried my hand at finding the root cause in the sklearn method but didn't have much success. However, I found out that the procedure causing the issue runs fine for what I assume are all but one iteration and then crashes on the last pass, but I was unable to figure out what might cause the difference that makes it crash. Maybe anyone here has an idea what causes this issue?
The error trace:
sorting = run_sorter(sorter.sorter_name, rec.Extractor, folder = folder, verbose = True, remove_existing_folder=overwrite, **params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ProgramData\mambaforge\envs\maxwelltesting\Lib\site-packages\spikeinterface\sorters\runsorter.py", line 199, in run_sorter
return run_sorter_local(**common_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ProgramData\mambaforge\envs\maxwelltesting\Lib\site-packages\spikeinterface\sorters\runsorter.py", line 261, in run_sorter_local
SorterClass.run_from_folder(folder, raise_error, verbose)
File "C:\ProgramData\mambaforge\envs\maxwelltesting\Lib\site-packages\spikeinterface\sorters\basesorter.py", line 301, in run_from_folder
raise SpikeSortingError(
spikeinterface.sorters.utils.misc.SpikeSortingError: Spike sorting error trace:
Traceback (most recent call last):
File "C:\ProgramData\mambaforge\envs\maxwelltesting\Lib\site-packages\spikeinterface\sorters\basesorter.py", line 261, in run_from_folder
SorterClass._run_from_folder(sorter_output_folder, sorter_params, verbose)
File "C:\ProgramData\mambaforge\envs\maxwelltesting\Lib\site-packages\spikeinterface\sorters\internal\spyking_circus2.py", line 152, in _run_from_folder
recording_w = whiten(recording_f, mode="local", radius_um=radius_um, dtype="float32", regularize=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ProgramData\mambaforge\envs\maxwelltesting\Lib\site-packages\spikeinterface\preprocessing\whiten.py", line 84, in __init__
W, M = compute_whitening_matrix(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ProgramData\mambaforge\envs\maxwelltesting\Lib\site-packages\spikeinterface\preprocessing\whiten.py", line 209, in compute_whitening_matrix
estimator.fit(data)
File "C:\ProgramData\mambaforge\envs\maxwelltesting\Lib\site-packages\sklearn\base.py", line 1473, in wrapper
return fit_method(estimator, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ProgramData\mambaforge\envs\maxwelltesting\Lib\site-packages\sklearn\covariance\_graph_lasso.py", line 1109, in fit
self.covariance_, self.precision_, self.costs_, self.n_iter_ = _graphical_lasso(
^^^^^^^^^^^^^^^^^
File "C:\ProgramData\mambaforge\envs\maxwelltesting\Lib\site-packages\sklearn\covariance\_graph_lasso.py", line 139, in _graphical_lasso
coefs, _, _, _ = cd_fast.enet_coordinate_descent_gram(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "_cd_fast.pyx", line 569, in sklearn.linear_model._cd_fast.enet_coordinate_descent_gram
ValueError: Buffer dtype mismatch, expected 'const double' but got 'float'
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at spikeinterface/sorters/internal/spyking_circus2.py, where whitening is called with dtype="float32", then inspect spikeinterface/preprocessing/whiten.py around compute_whitening_matrix and estimator.fit(data). Reproduce the MaxTwo path and trace the final iteration's dtype; done means the SpykingCircus2 whitening path no longer raises the sklearn buffer mismatch for this input.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, scikit-learn
- Domain
- data, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100