apache / apache/datafusion

`percentile_cont` truncates Float64 interpolation weights to six decimal places

Open Beginner friendly
#25,266 2 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Describe the bug

I found that `percentile_cont` quantizes its Float64 interpolation weight in steps of `1e-6`. Small nonzero weights therefore become zero, producing the lower input value instead of an interpolated result.

### To Reproduce

I reproduced this on 55.1.0 and `main` at `9082d6b10`.

```sql
SELECT percentile_cont(x, CAST(2.5e-7 AS DOUBLE))
FROM (VALUES
(CAST(0.0 AS DOUBLE)),
(CAST(1.0 AS DOUBLE)),
(CAST(2.0 AS DOUBLE))
) AS t(x);
```

```text
actual: 0.0
expected: 5e-7

### Expected behavior

The rank is `(3 - 1) * 2.5e-7 = 5e-7`, so linear interpolation between `0.0` and `1.0` should return approximately `5e-7`.

### Additional context

The Float64 path converts the fraction to an integer using [`INTERPOLATION_PRECISION = 1_000_000`](https://github.com/apache/datafusion/blob/9082d6b10c29b72d56bede3d8e353d9d61fde542/datafusion/functions-aggregate/src/percentile_cont.rs#L935-L945). That truncates this weight to zero. I couldn't find an existing issue for the Float64 behavior.

Contributor guide

Open the contributing guide

Research direction

Start in datafusion/functions-aggregate/src/percentile_cont.rs, especially the Float64 path around INTERPOLATION_PRECISION at lines 935–945. Run the SQL reproduction from the issue and trace how the interpolation weight is converted. Done when the query returns approximately 5e-7 instead of 0.0 for the reported input.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust, sql
Domain
databases
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
82/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.