apache / apache/arrow

[R] Update the docs to show how to avoid situations like data loss with leading zero in partition column

Open
#39,660 6 comments 0 reactions 0 assignees View on GitHub
Component: R good-first-issue Status: needs champion Type: enhancement
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

### Describe the bug, including details regarding any error messages, version, and platform.

Arrow does not correctly interpret some partition values as character columns, resulting in data loss.

``` r
library(arrow)
library(dplyr)

# character col with leading zero
mtcars2 <- mutate(mtcars, test = paste0(0, mtcars$carb)) |> as_tibble()

tmp <- tempfile()

write_dataset(mtcars2, tmp, partitioning = "test")

mtcars3 <- open_dataset(tmp) |> collect()

unique(mtcars2$test)
#> [1] "04" "01" "02" "03" "06" "08"
unique(mtcars3$test)
#> [1] 1 2 3 4 6 8
```

Created on 2024-01-17 with [reprex v2.0.2](https://reprex.tidyverse.org)

### Component(s)

R

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.