SpikeInterface / SpikeInterface/spikeinterface

Issue preprocessing: filtered traces after saving show artifacts

Open
#4,013 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
847
Forks
280
Avg merge
3d 9h
Merged PRs (30d)
29

Description

Hi, I'm having issues when preprocessing my recording and saving with recording.save function: the trace is not the same before and after saving, it seems that the saving function is introducing artifacts. This happens when applying a bandpass filter in my case.

I'm using spikeinterface version 0.101.2

import pathlib, os
import spikeinterface.extractors as se
import spikeinterface.preprocessing as spre
import spikeinterface.full as si
import matplotlib.pyplot as plt

selected_file_path = pathlib.Path("test.bin")

freq_min = 5
freq_max = 200

recording = se.read_axona(selected_file_path)
recordingf = spre.bandpass_filter(recording, freq_min=freq_min, freq_max=freq_max)

# Save
n_cpus = os.cpu_count()
n_jobs = n_cpus - 4
job_kwargs = dict(n_jobs=n_jobs, progress_bar=True)
save_filename = "save_filter_chunk_1s" 
saved_recording = recordingf.save(folder = save_filename, name = save_filename, **job_kwargs)

# Compare recording
reloaded_recording = si.load_extractor(save_filename)
original_trace = recordingf.get_traces()[:,0]
saved_trace = saved_recording.get_traces()[:,0]
reloaded_trace = reloaded_recording.get_traces()[:,0]

delta = original_trace - saved_trace
delta1 = original_trace - reloaded_trace
delta2 = saved_trace - reloaded_trace

fig, ax = plt.subplots(3, 1)
fig.set_figwidth(20)
fig.set_figheight(8)
ax[0].plot(delta)
ax[1].plot(delta1)
ax[2].plot(delta2)

Image

fig, ax = plt.subplots(3, 1)
fig.set_figwidth(20)
fig.set_figheight(8)
ax[0].plot(original_trace)
ax[1].plot(saved_trace)
ax[2].plot(reloaded_trace)

Image

I then tried with chunk_duration = recording.get_duration() since it seemed to have these artifacts every 1 second, and it seemed to work, but since chunk_duration default value is 1s maybe there's something in the saving after filtering that is not working properly?

save_filename = "save_filter_chunk_duration" 
saved_recording = recordingf.save(folder = save_filename, name = save_filename, chunk_duration = recording.get_duration(), **job_kwargs)

# Compare recordings
reloaded_recording = si.load_extractor(save_filename)
original_trace = recordingf.get_traces()[:,0]
saved_trace = saved_recording.get_traces()[:,0]
reloaded_trace = reloaded_recording.get_traces()[:,0]

delta = original_trace - saved_trace
delta1 = original_trace - reloaded_trace
delta2 = saved_trace - reloaded_trace

fig, ax = plt.subplots(3, 1)
fig.set_figwidth(20)
fig.set_figheight(8)
ax[0].plot(delta)
ax[1].plot(delta1)
ax[2].plot(delta2)

Image

Finally, I also tried to just load and save the recording without applying any preprocessing and it does not show this issue.
For reference I uploaded my scripts and data used here for your testing: https://drive.google.com/drive/folders/1h0clLnAmt0-R2TO4yP1rAJwXZW5WgrKC

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the provided reproduction with spikeinterface 0.101.2, focusing on bandpass_filter, recording.save, and load_extractor. Compare traces using the three deltas and test the default 1-second chunking against recording.get_duration(). Done means saving and reloading the filtered recording does not introduce the reported chunk-boundary artifacts.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.