SpikeInterface / SpikeInterface/spikeinterface
The `get_traces` method in `BaseRecording` should support assigning traces to previously allocated memory
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 847
- Forks
- 280
- Avg merge
- 3d 9h
- Merged PRs (30d)
- 29
Description
Let's focus on Neo as this will cover a lot of ground and then we can discuss the rest as edge cases / enchancements. For neo segments the traces look like this:
def get_traces(
self,
start_frame: Union[int, None] = None,
end_frame: Union[int, None] = None,
channel_indices: Union[List, None] = None,
) -> np.ndarray:
raw_traces = self.neo_reader.get_analogsignal_chunk(
block_index=self.block_index,
seg_index=self.segment_index,
i_start=start_frame,
i_stop=end_frame,
stream_index=self.stream_index,
channel_indexes=channel_indices,
)
return raw_traces
I think we should follow numpy ufuncs API and add and extra argument called out where we can pass a previously allocated memory array where the traces will be written. Example in numpy:
import numpy as np
# Pre-allocate an array of the desired shape
# Generate some data
data = np.array([1, 2, 3])
# Use numpy.square to fill 'out'
result = np.square(data, out=data)
np.shares_memory(data, result)
Reasons to do this:
- for our (power) users: we will give them more control over their memory allocations outsourcing optimization tricks to their side.
- Within our own code: this will help us avoid situations like the following where we have to sacrifice sequential readiability to avoid an undesirable memory allocation.
I think this makes the API more powerful and we already have numpy doing the work of providing a standard way of doing stuff. Most our users should be familiar with numpy and we can leverage on that for discoverability / usability.
As for implementation, for base recording get_traces is a read operation that ideally should return a memmap object or something of the like. With the out parameter we just make an assignation. A minimal check of shape can be added if we desire to add meaningul assertions for example.
What do you think guys? Are there downsides of this that I am not seeing?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with BaseRecording.get_traces and the Neo implementation that calls self.neo_reader.get_analogsignal_chunk. Review how start_frame, end_frame, and channel_indices determine the returned shape, then define the expected out-array behavior and shape checks. Done means Neo traces can be written into previously allocated memory while preserving the existing read behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend-api-design, data
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100