apache / apache/datafusion-comet

Remove is_valid_decimal_precision duplicated from arrow-rs

Open
#5,089 1 comment 0 reactions 1 assignee Claimed by @Smallfu666 View on GitHub
area:expressions enhancement
Dominant language
Scala
Stars
1.3k
Forks
373
Avg merge
2d 6h
Merged PRs (30d)
190

Description

### What is the problem the feature request solves?

`is_valid_decimal_precision` in `native/spark-expr/src/utils.rs` is a character-for-character copy of arrow's implementation (verified against the vendored arrow-data 58.4.0 source). The function's own comment says it can be removed once arrow-rs includes https://github.com/apache/arrow-rs/pull/6419, and that PR shipped long before the arrow 58 we depend on.

### Describe the potential solution

Delete the copy and use `arrow::datatypes::Decimal128Type::validate_decimal_precision(value, precision).is_ok()` (or the `is_valid_decimal_precision` trait method) at the call sites, which include `agg_funcs/sum_decimal.rs` and `agg_funcs/avg_decimal.rs`.

### Additional context

No semantic risk: the implementations are identical, including the bounds tables. The `precision <= DECIMAL128_MAX_PRECISION` guard is subsumed by arrow's validation.

Found during an audit of native code that replicates existing arrow-rs kernels.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.