dask / dask/crick

Memory leak in quantile function

Open
#23 4 comments 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
27
Forks
20
PR merge metrics
No merged PRs in 30d

Description

Running:
```
#!/usr/bin/env python3
import numpy as np
import datetime
from crick import TDigest

td = TDigest()
for j in range(1000000):
arr = np.array(1)
td.update(arr)
x = td.quantile(0.88)
```

Leads to this memory usage pattern:
![memoryleak-crick](https://user-images.githubusercontent.com/868434/100587332-b5e29f80-32f0-11eb-838e-d6ac684e1a5f.png)

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the reported loop with Python, NumPy, and crick's TDigest.quantile call, then inspect the quantile path and its memory behavior. Done means the million-iteration example no longer shows continuing memory growth while preserving the reported quantile behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, python
Domain
data, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.