apache / apache/datafusion-comet

Support expressions already implemented in datafusion-spark crate

Open
#4,150 1 comment 0 reactions 1 assignee Claimed by @comphead View on GitHub
area:expressions enhancement priority:medium
Dominant language
Scala
Stars
1.3k
Forks
373
Avg merge
2d 6h
Merged PRs (30d)
190

Description

29 Spark expressions are already implemented in the `datafusion-spark` crate but not yet wired up in Comet. Adding support for these is lower effort than typical expression work since the native implementation already exists — the main task is adding the JVM serde layer.

| Category | Expressions |
|---|---|
| aggregate | `collect_list`, `try_sum` |
| array | `shuffle`, `slice` |
| bitwise | `shiftrightunsigned` |
| datetime | `add_months`, `from_utc_timestamp`, `make_dt_interval`, `make_interval`, `to_utc_timestamp`, `unix_micros`, `unix_millis`, `unix_seconds` |
| json | `json_tuple` |
| map | `map_from_arrays`, `map_from_entries` |
| math | `csc`, `factorial`, `pmod`, `rint`, `sec` |
| string | `base64`, `elt`, `format_string`, `soundex`, `unbase64` |
| url | `parse_url`, `url_decode`, `url_encode` |

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.