SpikeInterface / SpikeInterface/spikeinterface
Shape mismatch error while applying curation
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 847
- Forks
- 280
- Avg merge
- 3d 9h
- Merged PRs (30d)
- 29
Description
Hello,
While applying curation using results = sif.apply_curation(sorting_Analayzer, curation_dict, merging_mode='hard'), I get the following error. After execution, both compute_waveforms and Compute : spike_amplitudes + spike_locations progress bars reach 100%, and only then I get the error.
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
Cell In[25], line 8
6 with open(savingPath + '\\Sorters\\Sorter_Analyzer_Kilosort4_PFC_VGA\\spikeinterface_gui\\curation_data.json', 'r') as file:
7 curation_dict_PFC = json.load(file)
----> 8 sorting_Analayzer_RR_Analyzer_MergedNRemoved_PFC = sif.apply_curation(sorting_Analayzer_RR_Analyzer_PFC, curation_dict_PFC, merging_mode='hard')
File c:\Users\Majid\anaconda3\envs\spikeinterface_env\Lib\site-packages\spikeinterface\curation\curation_format.py:354, in apply_curation(sorting_or_analyzer, curation_dict, censor_ms, new_id_strategy, merging_mode, sparsity_overlap, verbose, **job_kwargs)
342 analyzer = analyzer.remove_units(curation_dict["removed_units"])
343 analyzer, new_unit_ids = analyzer.merge_units(
344 curation_dict["merge_unit_groups"],
345 censor_ms=censor_ms,
(...)
352 **job_kwargs,
353 )
--> 354 apply_curation_labels(analyzer.sorting, new_unit_ids, curation_dict)
355 return analyzer
356 else:
File c:\Users\Majid\anaconda3\envs\spikeinterface_env\Lib\site-packages\spikeinterface\curation\curation_format.py:256, in apply_curation_labels(sorting, new_unit_ids, curation_dict)
253 group_values.append(value)
254 if len(set(group_values)) == 1:
255 # all group has the same label or empty
--> 256 sorting.set_property(key, values=group_values, ids=[new_unit_id])
257 else:
259 for key in label_def["label_options"]:
File c:\Users\Majid\anaconda3\envs\spikeinterface_env\Lib\site-packages\spikeinterface\core\base.py:296, in BaseExtractor.set_property(self, key, values, ids, missing_value)
291 assert (
292 dtype_kind == existing_property.kind
293 ), f"Mismatch between existing property dtype {existing_property.kind} and provided values dtype {dtype_kind}."
295 indices = self.ids_to_indices(ids)
--> 296 self._properties[key][indices] = values
297 else:
298 indices = self.ids_to_indices(ids)
ValueError: shape mismatch: value array of shape (2,) could not be broadcast to indexing result of shape (1,)
If it is relevant, I get the following warnings before the error pops up and while computing the waveforms and apike amplitudes:
c:\Users\Majid\anaconda3\envs\spikeinterface_env\Lib\site-packages\spikeinterface\qualitymetrics\misc_metrics.py:908: UserWarning: Some units have too few spikes : amplitude_cutoff is set to NaN
warnings.warn(f"Some units have too few spikes : amplitude_cutoff is set to NaN")
c:\Users\Majid\anaconda3\envs\spikeinterface_env\Lib\site-packages\numpy\core\_methods.py:206: RuntimeWarning: Degrees of freedom <= 0 for slice
ret = _var(a, axis=axis, dtype=dtype, out=out, ddof=ddof,
c:\Users\Majid\anaconda3\envs\spikeinterface_env\Lib\site-packages\numpy\core\_methods.py:163: RuntimeWarning: invalid value encountered in divide
arrmean = um.true_divide(arrmean, div, out=arrmean,
c:\Users\Majid\anaconda3\envs\spikeinterface_env\Lib\site-packages\numpy\core\_methods.py:198: RuntimeWarning: invalid value encountered in divide
ret = ret.dtype.type(ret / rcount)
I am using spikeinterface==0.101.2.
Thanks in advance for your assistant!
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the failure with spikeinterface==0.101.2 using apply_curation(..., merging_mode='hard'). Trace the path through curation_format.py's apply_curation and apply_curation_labels into BaseExtractor.set_property, focusing on the mismatch between group_values and the selected new_unit_id; done means curation completes without the shape-mismatch error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100