SpikeInterface / SpikeInterface/spikeinterface

noise caused by whitening

Open
#4,160 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

preprocessing
Dominant language
Python
Stars
847
Forks
280
Avg merge
3d 9h
Merged PRs (30d)
29

Description

Dear Spikeinterface community

My recording is about neurons with giant axons + its has a high spontaneous firing rate (40 Hz, can go up to 150 Hz in response to visual stimuli) so detecting its spike is easy. However, when doing spike sorting, I bumped into issues such as a waveform residual from one unit being clustered as a separate unit at other channels or noise of all kinds being clustered as units.
Here is an overview of spike sorting done by kilosort standalone (blue and red are the waveform residual from the same unit)
Image

I read about preprocessing steps https://github.com/SpikeInterface/spikeinterface/issues/2017, #3483, and from my understanding whitening can decorrelate data but can also cause noise (see this notebook for further info). Therefore, I was wondering if optimising the whitening step can improve the performance of spike sorting or whitening is suitable for this kind neural signals.

Then I compared the results of whitening from 4 different recording sessions at 4 different radius in the local mode
Here are the code and the graphs

r_range=[25,50,100,150]
for this_r in r_range:
    rec_w = spre.whiten(recording=recording_corrected,mode="local",radius_um=this_r,int_scale=200,dtype=float)
Image Image Image Image

In general, while some waveform residuals were removed, it looks like whitening introduced noise across channels.
Could anyone give me suggestion what would be the best way to optimise parameters of whitening or whether whitening is suitable for my dataset?

Note1: setting int_scale=200 is meant to replicate the kilosort behaviour. I have removed that parameter but the issue still exists.
Note2: the channels were sorted by their ID not depth or location
Note3: these acute recording were done using CambridgeNeurotech H10 probe (32-channel per shank within 330 um in depth, 18.5um in hozitonal distance and 30 um in vertical distance) in head-fixed insects (typically around 1 hour).
Note4: my preprocessing steps start from bandpass filter (600-6000Hz), detecting bad channels, common median reference, correction motion by DREDge to whitening step

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the spre.whiten call shown in the issue and compare the local-mode radius_um and int_scale settings against the linked preprocessing discussions and notebook. Reproduce the whitening behavior across the described sessions, then establish whether the added noise is expected or a defect; done means a documented recommendation or a reproducible fix.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.