SpikeInterface / SpikeInterface/spikeinterface

problem recumputing waveforms in a binary sorting analyzer

Open
#4,146 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

question
Dominant language
Python
Stars
847
Forks
280
Avg merge
3d 9h
Merged PRs (30d)
29

Description

dear devs. when using a sorting analyzer in memory mode, i can correct spike times and recompute the waveforms.
unit_peak_shifts= score.get_template_extremum_channel_peak_shift(sorting_analyzer, peak_sign= 'neg')
sorting=spost.align_sorting(sorting, unit_peak_shifts)

sorting_analyzer.compute("waveforms", ms_before=1.5, ms_after=1.5, **job_kwargs)
sorting_analyzer.compute("templates", operators=["average", "median", "std"],**job_kwargs)

but in binary mode, after computing the waveforms in the first time, i cannot recompute them:

--> 294 sorting_analyzer.compute("waveforms",  ms_before=1.5, ms_after=1.5, **job_kwargs)
    295 sorting_analyzer.compute("templates", operators=["average", "median", "std"],**job_kwargs)
    296 print('computing template simliarity and metrics')
File ~\vspy\Lib\site-packages\spikeinterface\core\sortinganalyzer.py:1620, in SortingAnalyzer.compute(self, input, save, extension_params, verbose, **kwargs)
   1573 """
   1574 Compute one extension or several extensiosn.
   1575 Internally calls compute_one_extension() or compute_several_extensions() depending on the input type.
   (...)
   1617
   1618 """
   1619 if isinstance(input, str):
-> 1620     return self.compute_one_extension(extension_name=input, save=save, verbose=verbose, **kwargs)
   1621 elif isinstance(input, dict):
   1622     params_, job_kwargs = split_job_kwargs(kwargs)
File ~\vspy\Lib\site-packages\spikeinterface\core\sortinganalyzer.py:1697, in SortingAnalyzer.compute_one_extension(self, extension_name, save, verbose, **kwargs)
   1694     assert ok, f"Extension {extension_name} requires {dependency_name} to be computed first"
   1696 extension_instance = extension_class(self)
-> 1697 extension_instance.set_params(save=save, **params)
   1698 if extension_class.need_job_kwargs:
   1699     extension_instance.run(save=save, verbose=verbose, **job_kwargs)
File ~\vspy\Lib\site-packages\spikeinterface\core\sortinganalyzer.py:2654, in AnalyzerExtension.set_params(self, save, **params)
   2651 # this ensure data is also deleted and corresponds to params
   2652 # this also ensure the group is created
   2653 if save:
-> 2654     self._reset_extension_folder()
   2656 params = self._set_params(**params)
   2657 self.params = params
File ~\vspy\Lib\site-packages\spikeinterface\core\sortinganalyzer.py:2600, in AnalyzerExtension._reset_extension_folder(self)
   2598     extension_folder = self._get_binary_extension_folder()
   2599     if extension_folder.is_dir():
-> 2600         shutil.rmtree(extension_folder)
   2601     extension_folder.mkdir(exist_ok=False, parents=True)
   2603 elif self.format == "zarr":
File ~\.conda\envs\cupyenv\Lib\shutil.py:787, in rmtree(path, ignore_errors, onerror, dir_fd)
    785     # can't continue even if onerror hook returns
    786     return
--> 787 return _rmtree_unsafe(path, onerror)
File ~\.conda\envs\cupyenv\Lib\shutil.py:634, in _rmtree_unsafe(path, onerror)
    632             os.unlink(fullname)
    633         except OSError:
--> 634             onerror(os.unlink, fullname, sys.exc_info())
    635 try:
    636     os.rmdir(path)
File ~\.conda\envs\cupyenv\Lib\shutil.py:632, in _rmtree_unsafe(path, onerror)
    630 else:
    631     try:
--> 632         os.unlink(fullname)
    633     except OSError:
    634         onerror(os.unlink, fullname, sys.exc_info())
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'r\\[gpfs.corp.brain.mpg.de](http://gpfs.corp.brain.mpg.de/)\\stem\\data\\project_hierarchy\\data\\afm16924\\240526\\trial0\\postscript\\sorting_analyzer_results\\extensions\\waveforms\\waveforms.npy'

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in spikeinterface/core/sortinganalyzer.py at SortingAnalyzer.compute_one_extension(), AnalyzerExtension.set_params(), and _reset_extension_folder(). Reproduce the binary-mode waveform recomputation on Windows using the reported sequence and inspect why waveforms.npy remains in use; done means recomputation completes without the PermissionError.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.