SpikeInterface / SpikeInterface/spikeinterface
Issue preprocessing: filtered traces after saving show artifacts
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 847
- Forks
- 280
- Avg merge
- 3d 9h
- Merged PRs (30d)
- 29
Description
Hi, I'm having issues when preprocessing my recording and saving with recording.save function: the trace is not the same before and after saving, it seems that the saving function is introducing artifacts. This happens when applying a bandpass filter in my case.
I'm using spikeinterface version 0.101.2
import pathlib, os
import spikeinterface.extractors as se
import spikeinterface.preprocessing as spre
import spikeinterface.full as si
import matplotlib.pyplot as plt
selected_file_path = pathlib.Path("test.bin")
freq_min = 5
freq_max = 200
recording = se.read_axona(selected_file_path)
recordingf = spre.bandpass_filter(recording, freq_min=freq_min, freq_max=freq_max)
# Save
n_cpus = os.cpu_count()
n_jobs = n_cpus - 4
job_kwargs = dict(n_jobs=n_jobs, progress_bar=True)
save_filename = "save_filter_chunk_1s"
saved_recording = recordingf.save(folder = save_filename, name = save_filename, **job_kwargs)
# Compare recording
reloaded_recording = si.load_extractor(save_filename)
original_trace = recordingf.get_traces()[:,0]
saved_trace = saved_recording.get_traces()[:,0]
reloaded_trace = reloaded_recording.get_traces()[:,0]
delta = original_trace - saved_trace
delta1 = original_trace - reloaded_trace
delta2 = saved_trace - reloaded_trace
fig, ax = plt.subplots(3, 1)
fig.set_figwidth(20)
fig.set_figheight(8)
ax[0].plot(delta)
ax[1].plot(delta1)
ax[2].plot(delta2)
fig, ax = plt.subplots(3, 1)
fig.set_figwidth(20)
fig.set_figheight(8)
ax[0].plot(original_trace)
ax[1].plot(saved_trace)
ax[2].plot(reloaded_trace)
I then tried with chunk_duration = recording.get_duration() since it seemed to have these artifacts every 1 second, and it seemed to work, but since chunk_duration default value is 1s maybe there's something in the saving after filtering that is not working properly?
save_filename = "save_filter_chunk_duration"
saved_recording = recordingf.save(folder = save_filename, name = save_filename, chunk_duration = recording.get_duration(), **job_kwargs)
# Compare recordings
reloaded_recording = si.load_extractor(save_filename)
original_trace = recordingf.get_traces()[:,0]
saved_trace = saved_recording.get_traces()[:,0]
reloaded_trace = reloaded_recording.get_traces()[:,0]
delta = original_trace - saved_trace
delta1 = original_trace - reloaded_trace
delta2 = saved_trace - reloaded_trace
fig, ax = plt.subplots(3, 1)
fig.set_figwidth(20)
fig.set_figheight(8)
ax[0].plot(delta)
ax[1].plot(delta1)
ax[2].plot(delta2)
Finally, I also tried to just load and save the recording without applying any preprocessing and it does not show this issue.
For reference I uploaded my scripts and data used here for your testing: https://drive.google.com/drive/folders/1h0clLnAmt0-R2TO4yP1rAJwXZW5WgrKC
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by running the provided reproduction with spikeinterface 0.101.2, focusing on bandpass_filter, recording.save, and load_extractor. Compare traces using the three deltas and test the default 1-second chunking against recording.get_duration(). Done means saving and reloading the filtered recording does not introduce the reported chunk-boundary artifacts.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100