apache / apache/arrow

[C++] Error when creating Expression on Decimal128 types: precision out of range

Open
#32,848 11 comments 0 reactions 0 assignees View on GitHub
Component: C++ Type: bug
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 20h
Merged PRs (30d)
94

Description

Reproducer in R:

```Java

library(arrow)
library(dplyr)

tab <- Table$create(col1 = 1:4, col2 = 5:8)
tab <- tab$cast(schema(col1 = decimal128(33, 4), col2 = decimal128(15, 2)))
tab %>% mutate(col1 * col2)

# Error: Invalid: Decimal precision out of range [1, 38]: 49
# /Users/me/arrow/cpp/src/arrow/compute/kernels/scalar_arithmetic.cc:1078 DecimalType::Make(left_type.id(), precision, scale)
# /Users/me/arrow/cpp/src/arrow/compute/exec/expression.cc:413 call.kernel->signature->out_type().Resolve(&kernel_context, types)
```

We don't have this problem integers and floats (see comment below). For consistency with the other arithmetic functions, what I would expect would be that we would expand the precision as much as we could within Decimal128–in this case, Decimal128(38, 6)–and the compute function would either error _if_ there is an overflow (in the _checked version) or just overflow in the non-checked version. But it wouldn't error on determining the output type.

**Reporter**: [Neal Richardson](https://issues.apache.org/jira/browse/ARROW-17601) / @nealrichardson

**Note**: *This issue was originally created as [ARROW-17601](https://issues.apache.org/jira/browse/ARROW-17601). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*

Contributor guide

Open the contributing guide

Research direction

Start with the R reproducer and inspect cpp/src/arrow/compute/kernels/scalar_arithmetic.cc around the DecimalType::Make call and cpp/src/arrow/compute/exec/expression.cc around output-type resolution. Compare decimal arithmetic with integer and float behavior. Done means Decimal128 output precision is expanded within the allowed range and errors occur only for overflow in the checked operation, not during output-type determination.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, r
Domain
data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.