apache / apache/datafusion

Make the `tdigest` public

Open
#8,554 4 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Is your feature request related to a problem or challenge?

I would like to build a custom aggregator based on the [tdigest](https://github.com/apache/arrow-datafusion/blob/main/datafusion/physical-expr/src/aggregate/tdigest.rs). The tdigest `mod` is private, and thus I can't use it outside DataFusion. It would be great if it would be changed to poublic to allow custom aggregators based on it.

### Describe the solution you'd like

To allow building new aggregators based on the `tdigest` make it public by changing the following
https://github.com/apache/arrow-datafusion/blob/b276d479918400105017db1f7f46dcb67b52206d/datafusion/physical-expr/src/aggregate/mod.rs#L58
to
```rust
pub mod tdigest
```
and change the visibility in the tdigest `mod` to public too.

### Describe alternatives you've considered

If changing the visibility to public is not possible, I would need to introduce my own tdigest and reproduce a lot of code to make it work as an aggregator. This also means that I need to update the code with each new version of DataFusion.

### Additional context

_No response_

Contributor guide

Open the contributing guide

Research direction

Start by reading datafusion/physical-expr/src/aggregate/mod.rs at the referenced tdigest module declaration, then inspect datafusion/physical-expr/src/aggregate/tdigest.rs for the visibility changes requested. Done means external custom aggregators can access the tdigest module and its required public items, with the project checks still passing.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
databases
Issue type
Feature
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.