apache / apache/datafusion

Support 2-argument ceil(decimal, scale) for Spark

Open
#22,560 3 comments 0 reactions 1 assignee Claimed by @diegoQuinas View on GitHub
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Is your feature request related to a problem or challenge?

Follow-up to #21560 / #21710.

#21710 added 2-argument `ceil(value, scale)` to `datafusion-spark` for integer and floating-point inputs. The decimal path was deliberately left out of scope: `ceil(Decimal128(p, s), scale)` currently errors at planning time with `"2-argument ceil is not yet supported for decimal inputs"`.

Spark supports this form for decimal inputs and derives the result `(precision, scale)` from the literal `scale` argument, matching `RoundCeil` semantics. For example:

- `ceil(3.1411::decimal(5,4), 3)` → `decimal(5, 3)` value `3.142`
- `ceil(3.1411::decimal(5,4), -3)` → `decimal(4, 0)` value `1000`

### Describe the solution you'd like

Implement the decimal path in `SparkCeil`:

- Use `return_field_from_args` so the literal `scale` can drive the output `(precision, scale)`.
- When `scale` is not visible as a literal at plan time (column, non-literal cast), fall back to a conservative `Decimal128(min(p + 1, 38), s)`, same approach as `datafusion/functions/round`.
- Execution: operate on the unscaled integer, then rescale to the advertised output scale.
- Follow Spark's wrapping behavior on overflow (no error), matching the existing 1-arg `Decimal128` path.

### Describe alternatives you've considered

_No response_

### Additional context

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.