SpikeInterface / SpikeInterface/spikeinterface

Shape mismatch error while applying curation

Open
#3,579 11 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

curation
Dominant language
Python
Stars
847
Forks
280
Avg merge
3d 9h
Merged PRs (30d)
29

Description

Hello,

While applying curation using results = sif.apply_curation(sorting_Analayzer, curation_dict, merging_mode='hard'), I get the following error. After execution, both compute_waveforms and Compute : spike_amplitudes + spike_locations progress bars reach 100%, and only then I get the error.

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[25], line 8
      6 with open(savingPath + '\\Sorters\\Sorter_Analyzer_Kilosort4_PFC_VGA\\spikeinterface_gui\\curation_data.json', 'r') as file:
      7         curation_dict_PFC = json.load(file)
----> 8 sorting_Analayzer_RR_Analyzer_MergedNRemoved_PFC = sif.apply_curation(sorting_Analayzer_RR_Analyzer_PFC, curation_dict_PFC, merging_mode='hard')

File c:\Users\Majid\anaconda3\envs\spikeinterface_env\Lib\site-packages\spikeinterface\curation\curation_format.py:354, in apply_curation(sorting_or_analyzer, curation_dict, censor_ms, new_id_strategy, merging_mode, sparsity_overlap, verbose, **job_kwargs)
    342     analyzer = analyzer.remove_units(curation_dict["removed_units"])
    343     analyzer, new_unit_ids = analyzer.merge_units(
    344         curation_dict["merge_unit_groups"],
    345         censor_ms=censor_ms,
   (...)
    352         **job_kwargs,
    353     )
--> 354     apply_curation_labels(analyzer.sorting, new_unit_ids, curation_dict)
    355     return analyzer
    356 else:

File c:\Users\Majid\anaconda3\envs\spikeinterface_env\Lib\site-packages\spikeinterface\curation\curation_format.py:256, in apply_curation_labels(sorting, new_unit_ids, curation_dict)
    253             group_values.append(value)
    254     if len(set(group_values)) == 1:
    255         # all group has the same label or empty
--> 256         sorting.set_property(key, values=group_values, ids=[new_unit_id])
    257 else:
    259     for key in label_def["label_options"]:

File c:\Users\Majid\anaconda3\envs\spikeinterface_env\Lib\site-packages\spikeinterface\core\base.py:296, in BaseExtractor.set_property(self, key, values, ids, missing_value)
    291         assert (
    292             dtype_kind == existing_property.kind
    293         ), f"Mismatch between existing property dtype {existing_property.kind} and provided values dtype {dtype_kind}."
    295     indices = self.ids_to_indices(ids)
--> 296     self._properties[key][indices] = values
    297 else:
    298     indices = self.ids_to_indices(ids)

ValueError: shape mismatch: value array of shape (2,) could not be broadcast to indexing result of shape (1,)

If it is relevant, I get the following warnings before the error pops up and while computing the waveforms and apike amplitudes:

c:\Users\Majid\anaconda3\envs\spikeinterface_env\Lib\site-packages\spikeinterface\qualitymetrics\misc_metrics.py:908: UserWarning: Some units have too few spikes : amplitude_cutoff is set to NaN
  warnings.warn(f"Some units have too few spikes : amplitude_cutoff is set to NaN")
c:\Users\Majid\anaconda3\envs\spikeinterface_env\Lib\site-packages\numpy\core\_methods.py:206: RuntimeWarning: Degrees of freedom <= 0 for slice
  ret = _var(a, axis=axis, dtype=dtype, out=out, ddof=ddof,
c:\Users\Majid\anaconda3\envs\spikeinterface_env\Lib\site-packages\numpy\core\_methods.py:163: RuntimeWarning: invalid value encountered in divide
  arrmean = um.true_divide(arrmean, div, out=arrmean,
c:\Users\Majid\anaconda3\envs\spikeinterface_env\Lib\site-packages\numpy\core\_methods.py:198: RuntimeWarning: invalid value encountered in divide
  ret = ret.dtype.type(ret / rcount)

I am using spikeinterface==0.101.2.

Thanks in advance for your assistant!

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the failure with spikeinterface==0.101.2 using apply_curation(..., merging_mode='hard'). Trace the path through curation_format.py's apply_curation and apply_curation_labels into BaseExtractor.set_property, focusing on the mismatch between group_values and the selected new_unit_id; done means curation completes without the shape-mismatch error.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.