apache / apache/datafusion-comet
Support expressions already implemented in datafusion-spark crate
- Dominant language
- Scala
- Stars
- 1.3k
- Forks
- 373
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 190
Description
29 Spark expressions are already implemented in the `datafusion-spark` crate but not yet wired up in Comet. Adding support for these is lower effort than typical expression work since the native implementation already exists — the main task is adding the JVM serde layer.
| Category | Expressions |
|---|---|
| aggregate | `collect_list`, `try_sum` |
| array | `shuffle`, `slice` |
| bitwise | `shiftrightunsigned` |
| datetime | `add_months`, `from_utc_timestamp`, `make_dt_interval`, `make_interval`, `to_utc_timestamp`, `unix_micros`, `unix_millis`, `unix_seconds` |
| json | `json_tuple` |
| map | `map_from_arrays`, `map_from_entries` |
| math | `csc`, `factorial`, `pmod`, `rint`, `sec` |
| string | `base64`, `elt`, `format_string`, `soundex`, `unbase64` |
| url | `parse_url`, `url_decode`, `url_encode` |
Contributor guide
Assessment
This issue has not been assessed yet.