apache / apache/arrow-rs

add rounding logic and scale zero fix in parse_decimal to match parse_string_to_decimal_native behavior

Open
#7,355 1 comment 0 reactions 1 assignee Claimed by @neilconway View on GitHub
enhancement
Dominant language
Rust
Stars
3.6k
Forks
1.3k
Avg merge
2d 14h
Merged PRs (30d)
167

Description

- Existing string to decimal cast uses parse_string_to_decimal_native
- parse_string_to_decimal_native does not have support for e-notation
- parse_string_to_decimal_native does rounding at scale, not truncate
- parse_decimal an existing method has e-notation support and use elsewhere like arrow-csv, arrow-json.
- parse_decimal tests with scale 0, fails as shown below
```rust
assert_eq!(
parse_decimal::("123.45", 38, 0)?,
123_i128
);

assertion `left == right` failed
Left: 12345
Right: 123

```
- existing test using parse_decimal native works

```rust
assert_eq!(
parse_string_to_decimal_native::("123.45", 0)?,
123_i128
);
//works fine..
```

- pars_decimal truncates as defualt behavior, parse_string_to_decimal_native does rounding, hence read from json, or csv of a decimal has truncated value but result of a cast operation has rounded value.

```rust
assert_eq!(
parse_decimal::("123.4567891", 38, 5)?,
12345679_i128
);

assertion `left == right` failed
left: 12345678
right: 12345679

```
vs

```
assert_eq!(
parse_string_to_decimal_native::("123.4567891", 5)?,
12345679_i128
);

// works fine..
```

This issue was raised as part of investigation to fix this [datafusion issue] (https://github.com/apache/datafusion/issues/10315)

if parse_decimal adds rounding logic and fix the issue with scale=0, parse decimal can be used for string to decimal cast, thereby getting the scientific notation along with it.

parse_decimal internally calls parse_e_notation, so rounding logic needs to be added there too, so that

```rust
"12345e-5" with scale 4, yields 0.1235
"1265E-4" with scale 3 yields .127
```

**Describe the solution you'd like**

**Describe alternatives you've considered**

**Additional context**

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.