[C++][Python] Adding data to tdigest in pyarrow
- Dominant language
- C++
- Stars
- 17.1k
- Forks
- 4.3k
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 91
Description
### Describe the enhancement requested
Requests for the python implementation of the t-digest algorithm:
1. Currently it is only possible to compute one t-digest from a data array and return a given percentile. There doesn't seem to be an option to continuously 'add' data to the digest, as described in the clustering algorithm 2: https://arxiv.org/pdf/1902.04023.pdf. However I think this is happening under the hood, as reducing the buffer size, I assume, converts it the clustering algorithm variant. Would it be possible to include a a function such as pyarrow.compute.tdigest.add(), where you can add data points incrementally (as in the case of data streaming)?
2. Is it possible to alter the scale function used in the implementation? I'm not sure which scale function you have implemented but it would be nice to have some control over this!
Thanks
### Component(s)
Python
Contributor guide
Assessment
This issue has not been assessed yet.