SpikeInterface / SpikeInterface/spikeinterface
run_node_pipeline() should skip chunks without peaks
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 847
- Forks
- 280
- Avg merge
- 3d 9h
- Merged PRs (30d)
- 29
Description
I discussed that with Sam @samuelgarcia , but I think that currently, in SI, each time we have a node pipeline (or that we use the ChunkRecordingExecutor), this loads all the chunks to apply some internal function. However, if the first node is a PeakRetriever, one could and should first check that there is indeed peaks in a given chunk. Because otherwise, no need to load the traces, this is a waste or IO, and I guess there could be some speedup there
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating run_node_pipeline() and ChunkRecordingExecutor, then trace how chunks are loaded when the first node is a PeakRetriever. Determine how a chunk with no peaks can be identified before loading traces, and add coverage showing that such chunks are skipped without changing results or processing chunks that contain peaks.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- performance
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100