apache / apache/datafusion

Window functions with `ORDER BY` on a Duration / Interval / Struct / Map column fail with "Internal error: Cannot run range queries"

Open
#24,915 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Describe the bug

`OVER (ORDER BY x)` defaults to `RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW`, a "free range" frame with no offsets. Such a frame only needs the ORDER BY values to be comparable, but `coerce_window_frame` (`datafusion/optimizer/src/analyzer/type_coercion.rs`) calls `extract_window_frame_target_type` before checking `free_range()`, and that helper returns an *internal* error for every type without arithmetic.

So any window function ordered by a Duration, Interval, Struct or Map column fails, even `row_number()` or `count(*)`, and the message tells the user to file a bug.

### To Reproduce

```sql
SELECT x, row_number() OVER (ORDER BY x)
FROM (VALUES (arrow_cast(2, 'Duration(Second)')), (arrow_cast(1, 'Duration(Second)'))) t(x);
```

```text
Internal error: Cannot run range queries on datatype: Duration(s).
This issue was likely caused by a bug in DataFusion's code. Please help us to resolve this by filing a bug report in our issue tracker: https://github.com/apache/datafusion/issues
```

Same for `interval '1 month'`, `struct(1)` and `map(['a'], [1])` values; `window.slt` even pins the internal-error text for the struct case.

### Expected behavior

The query runs (the values sort fine), like it does for a Time column since the free-range handling for those types was added. A frame with a finite offset (`RANGE BETWEEN 1 PRECEDING ...`) over such a type should keep failing, but as a planning error.

### Additional context

Found while running a cast matrix of extreme values against a debug build of `datafusion-cli`.

Contributor guide

Open the contributing guide

Research direction

Start in datafusion/optimizer/src/analyzer/type_coercion.rs, at coerce_window_frame and extract_window_frame_target_type, then reproduce the Duration query with a debug datafusion-cli build. Review window.slt, including the struct case, and verify that free-range frames run for Duration, Interval, Struct, and Map while finite-offset frames still produce a planning error.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust, sql
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
76/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.