SpikeInterface / SpikeInterface/spikeinterface

possible memory leak in lsqr

Open
#3,128 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

motion correction
Dominant language
Python
Stars
847
Forks
280
Avg merge
3d 9h
Merged PRs (30d)
29

Description

In motion estimation global displacement lsqr robust I encountered OOM issues, similar to those encountered with the lsmr. Maybe a leak. I also fixed with gc.collect() but for some reason I had issues with the import. This is my code block:

        # TODO: this is already soft_weights
        import gc
        xrange = trange if progress_bar else range
        for i in xrange(lsqr_robust_n_iter):
            p = lsqr(A[idx].multiply(W[idx]), V[idx] * W[idx][:, 0])[0]
            idx = np.nonzero(np.abs(zscore(A @ p - V)) <= robust_regression_sigma)
            import gc
            gc.collect()
            # print('temporary memory cleared')
        displacement = p
        gc.collect()
        print('temporary memory cleared')

I never bothered to figure out which of these two did the job. Just an FYI

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Locate the lsqr implementation used by motion estimation global displacement and reproduce the OOM with the reported robust-iteration loop. Compare memory behavior with and without gc.collect(); done means identifying whether a leak exists and documenting or testing a fix, but the report names no source file or test.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.