add rounding logic and scale zero fix in parse_decimal to match parse_string_to_decimal_native behavior
- Dominant language
- Rust
- Stars
- 3.6k
- Forks
- 1.3k
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 167
Description
- Existing string to decimal cast uses parse_string_to_decimal_native
- parse_string_to_decimal_native does not have support for e-notation
- parse_string_to_decimal_native does rounding at scale, not truncate
- parse_decimal an existing method has e-notation support and use elsewhere like arrow-csv, arrow-json.
- parse_decimal tests with scale 0, fails as shown below
```rust
assert_eq!(
parse_decimal::("123.45", 38, 0)?,
123_i128
);
assertion `left == right` failed
Left: 12345
Right: 123
```
- existing test using parse_decimal native works
```rust
assert_eq!(
parse_string_to_decimal_native::("123.45", 0)?,
123_i128
);
//works fine..
```
- pars_decimal truncates as defualt behavior, parse_string_to_decimal_native does rounding, hence read from json, or csv of a decimal has truncated value but result of a cast operation has rounded value.
```rust
assert_eq!(
parse_decimal::("123.4567891", 38, 5)?,
12345679_i128
);
assertion `left == right` failed
left: 12345678
right: 12345679
```
vs
```
assert_eq!(
parse_string_to_decimal_native::("123.4567891", 5)?,
12345679_i128
);
// works fine..
```
This issue was raised as part of investigation to fix this [datafusion issue] (https://github.com/apache/datafusion/issues/10315)
if parse_decimal adds rounding logic and fix the issue with scale=0, parse decimal can be used for string to decimal cast, thereby getting the scientific notation along with it.
parse_decimal internally calls parse_e_notation, so rounding logic needs to be added there too, so that
```rust
"12345e-5" with scale 4, yields 0.1235
"1265E-4" with scale 3 yields .127
```
**Describe the solution you'd like**
**Describe alternatives you've considered**
**Additional context**
Contributor guide
Assessment
This issue has not been assessed yet.