apache / apache/datafusion-comet
Support GROUPS window frame units
- Dominant language
- Scala
- Stars
- 1.3k
- Forks
- 373
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 198
Description
Comet supports `ROWS` and `RANGE` window frame units but not `GROUPS`.
The proto enum only defines `Rows` and `Range` (`native/proto/src/proto/operator.proto`, `WindowFrameType`), and the native planner has explicit guards that return "WindowFrameUnits::Groups is not supported." (`native/core/src/execution/planner.rs`, in `create_window_expr`).
Spark supports `GROUPS` frames, so queries using them fall back to Spark. Adding support means extending the proto enum, the Scala serde in `CometWindowExec.scala`, and the native frame construction in `planner.rs`.
Documented as a current limitation in the window function compatibility guide.
Contributor guide
Research direction
Start with WindowFrameType in native/proto/src/proto/operator.proto, then trace the Scala serialization in CometWindowExec.scala and create_window_expr in native/core/src/execution/planner.rs. Compare the existing ROWS and RANGE handling before addressing the GROUPS guard. Done means GROUPS queries are supported without falling back to Spark and the compatibility guide no longer lists this limitation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust, scala, spark
- Domain
- backend, data-engineering
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 55/100