apache / apache/datafusion-comet
Support RANGE window frames with an explicit offset on DATE or DECIMAL ORDER BY
- Dominant language
- Scala
- Stars
- 1.3k
- Forks
- 373
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 198
Description
Comet accelerates window functions with `RANGE` frames, but falls back to Spark when the frame uses an explicit `PRECEDING`/`FOLLOWING` offset and the `ORDER BY` column is `DATE` or `DECIMAL`.
See the fallbacks in `spark/src/main/scala/org/apache/spark/sql/comet/CometWindowExec.scala` (the RANGE-frame checks around lines 377-391):
- DATE ORDER BY: falls back with "RANGE frame with explicit offset on DATE ORDER BY is not supported" because Date32 + Int32 frame arithmetic is not handled in the native comparator.
- DECIMAL ORDER BY: falls back with "RANGE frame with explicit offset on DECIMAL ORDER BY is not supported" because decimal precision widening breaks the frame comparator.
Pure `UNBOUNDED`/`CURRENT ROW` RANGE frames on these order-by types already run natively; only explicit numeric offsets fall back.
Documented as a current limitation in the window function compatibility guide.
Contributor guide
Research direction
Start with the RANGE-frame checks around lines 377-391 in spark/src/main/scala/org/apache/spark/sql/comet/CometWindowExec.scala and trace the native comparator path for DATE and DECIMAL ORDER BY. Use the window function compatibility guide and existing window-function coverage to verify that explicit PRECEDING/FOLLOWING offsets run natively while UNBOUNDED and CURRENT ROW behavior remains unchanged.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- scala, spark
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100