apache / apache/arrow

[C++] Cast float to decimal truncates

Open
#32,852 7 comments 0 reactions 0 assignees View on GitHub
Component: C++ Type: bug
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 18h
Merged PRs (30d)
91

Description

Even with all of the cast options to allow truncation and overflow set to False, cast from double to decimal truncates.

Short reproducer in R:

```Java

library(arrow, warn.conflicts = FALSE)

a <- Scalar$create(sqrt(2))
a
#> Scalar
#> 1.4142135623730951

a$cast(decimal(5, 1))
#> Scalar
#> 1.4
```

Longer version that shows the cast options:

```Java

library(arrow, warn.conflicts = FALSE)
library(dplyr, warn.conflicts = FALSE)

tab <- arrow_table(a = sqrt(2))
tab
#> Table
#> 1 rows x 1 columns
#> $a

tab$a
#> ChunkedArray
#>
#> [
#> [
#> 1.4142135623730951
#> ]
#> ]

# See that all of the cast options are false:
tab %>%
mutate(b = cast(a, decimal(5, 1)))
#> Table (query)
#> a: double
#> b: decimal128(5, 1) (cast(a, {to_type=decimal128(5, 1), allow_int_overflow=false, allow_time_truncate=false, allow_time_overflow=false, allow_decimal_truncate=false, allow_float_truncate=false, allow_invalid_utf8=false}))
#>
#> See $.data for the source Arrow object

new_tab <- tab %>%
mutate(b = cast(a, decimal(5, 1))) %>%
compute()
new_tab
#> Table
#> 1 rows x 2 columns
#> $a
#> $b

new_tab$b
#> ChunkedArray
#>
#> [
#> [
#> 1.4
#> ]
#> ]
```

**Reporter**: [Neal Richardson](https://issues.apache.org/jira/browse/ARROW-17606) / @nealrichardson

**Note**: *This issue was originally created as [ARROW-17606](https://issues.apache.org/jira/browse/ARROW-17606). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*

Contributor guide

Open the contributing guide

Research direction

Run the supplied R reproducer and inspect the C++ cast entry point for double-to-decimal handling, including the listed cast options. Done means casting with allow_float_truncate=false and allow_decimal_truncate=false no longer silently produces 1.4, with coverage for the reported behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, r
Domain
data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.