apache / apache/arrow-rs

Casting decimals that exceed declared precision is buggy

Open
#10,947 1 comment 0 reactions 1 assignee Claimed by @neilconway View on GitHub
bug
Dominant language
Rust
Stars
3.6k
Forks
1.3k
Avg merge
2d 16h
Merged PRs (30d)
168

Description

### Describe the bug

A decimal can hold values that exceed its declared precision. Casting such a value to another decimal type either panics — even with safe = true, which is documented to replace unrepresentable values with null — or silently returns a wrapped value in both safe and strict modes, depending on the target type.

### To Reproduce

```rust
use arrow_array::Decimal128Array;
use arrow_cast::{cast_with_options, CastOptions};
use arrow_schema::DataType;

let array = Decimal128Array::from(vec![10_i128.pow(37)])
.with_precision_and_scale(5, 0)
.unwrap();

let result = cast_with_options(&array, &DataType::Decimal128(9, 2), &options).unwrap();
// safe=true and safe=false both return incorrect raw value
// -20847100762815390390123822295304634368
// and do not indicate an error
```

Related / similar:

```rust
use arrow_array::Decimal128Array;
use arrow_cast::{cast_with_options, CastOptions};
use arrow_schema::DataType;

// Declared precision 2; the stored value has 31 digits
let array = Decimal128Array::from(vec![10_i128.pow(30)])
.with_precision_and_scale(2, 1)
.unwrap();

let result = cast_with_options(
&array,
&DataType::Decimal32(9, 2),
&CastOptions { safe: true, ..Default::default() },
);
```

with `safe=true`, this results in

```
thread 'main' panicked at arrow-cast/src/cast/decimal.rs:193:56:
called `Option::unwrap()` on a `None` value
```

### Expected behavior

Values that cannot be represented in the target type become null in safe mode and an error with safe = false.

### Additional context

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.