SpikeInterface / SpikeInterface/spikeinterface

Save the preprocessing pipelines for simple reuse in curation GUI

Open
#1,103 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Python
Stars
847
Forks
280
Avg merge
3d 9h
Merged PRs (30d)
29

Description

Since we are now benchmarking drift registration, and that results starts to work, I was wondering if we could start thinking about a way to properly save a "preprocessing" pipeline, such that it could be reloaded and applied on the fly to the data. This issue came up after thinking about the manual curation step in a world where data would have been distorted via motion correction. Ideally, one would like to be able to view, dynamically, the data before and after motion correction. So if I have a pre-processing chain, let's say filtering -> destriping -> CAR -> z_score -> motion correction, could it be possible to save it (alongside with all the input argument) such that, in the curation step, one could load the recording, create again this lazy pre-processing chains and apply it (or not) on the fly?
We want to avoid data duplication, but also to appreciate the effect of the motion correction, and spot possible problems, it would be good to be able to turn it on or off in a GUI or in ipywidgets. Of course, when you know the chain, you can create several intermediate recordings and visualize each of them individually, but for complex chains this can start to be tedious.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating the existing preprocessing pipeline construction and the curation GUI or ipywidgets entry points, then inspect how lazy recordings and preprocessing arguments are represented. The design should support saving and reloading a chain such as filtering, destriping, CAR, z_score, and motion correction, applying it on demand without duplicating data, and toggling it for before-and-after visualization.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.