Support 2-argument ceil(decimal, scale) for Spark
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
### Is your feature request related to a problem or challenge?
Follow-up to #21560 / #21710.
#21710 added 2-argument `ceil(value, scale)` to `datafusion-spark` for integer and floating-point inputs. The decimal path was deliberately left out of scope: `ceil(Decimal128(p, s), scale)` currently errors at planning time with `"2-argument ceil is not yet supported for decimal inputs"`.
Spark supports this form for decimal inputs and derives the result `(precision, scale)` from the literal `scale` argument, matching `RoundCeil` semantics. For example:
- `ceil(3.1411::decimal(5,4), 3)` → `decimal(5, 3)` value `3.142`
- `ceil(3.1411::decimal(5,4), -3)` → `decimal(4, 0)` value `1000`
### Describe the solution you'd like
Implement the decimal path in `SparkCeil`:
- Use `return_field_from_args` so the literal `scale` can drive the output `(precision, scale)`.
- When `scale` is not visible as a literal at plan time (column, non-literal cast), fall back to a conservative `Decimal128(min(p + 1, 38), s)`, same approach as `datafusion/functions/round`.
- Execution: operate on the unscaled integer, then rescale to the advertised output scale.
- Follow Spark's wrapping behavior on overflow (no error), matching the existing 1-arg `Decimal128` path.
### Describe alternatives you've considered
_No response_
### Additional context
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.