apache / apache/datafusion-comet

Support GROUPS window frame units

Open
#4,836 2 comments 0 reactions 0 assignees View on GitHub
area:expressions enhancement
Dominant language
Scala
Stars
1.3k
Forks
373
Avg merge
2d 4h
Merged PRs (30d)
198

Description

Comet supports `ROWS` and `RANGE` window frame units but not `GROUPS`.

The proto enum only defines `Rows` and `Range` (`native/proto/src/proto/operator.proto`, `WindowFrameType`), and the native planner has explicit guards that return "WindowFrameUnits::Groups is not supported." (`native/core/src/execution/planner.rs`, in `create_window_expr`).

Spark supports `GROUPS` frames, so queries using them fall back to Spark. Adding support means extending the proto enum, the Scala serde in `CometWindowExec.scala`, and the native frame construction in `planner.rs`.

Documented as a current limitation in the window function compatibility guide.

Contributor guide

Open the contributing guide

Research direction

Start with WindowFrameType in native/proto/src/proto/operator.proto, then trace the Scala serialization in CometWindowExec.scala and create_window_expr in native/core/src/execution/planner.rs. Compare the existing ROWS and RANGE handling before addressing the GROUPS guard. Done means GROUPS queries are supported without falling back to Spark and the compatibility guide no longer lists this limitation.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust, scala, spark
Domain
backend, data-engineering
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.