apache / apache/arrow

[R] Error using `copy_files()`: "cannot use file ... as a directory"

Open
#35,994 3 comments 0 reactions 0 assignees View on GitHub
Component: R good-second-issue Priority: Critical Type: bug
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

### Describe the bug, including details regarding any error messages, version, and platform.

``` r
library(arrow)
uri <- "gs://anonymous@voltrondata-labs-datasets/diamonds/cut=Good/part-0.parquet"
# reading from the URI works
df <- read_parquet(uri)
df
#> carat color clarity depth table price x y z
#> 1 0.23 E VS1 56.9 65.0 327 4.05 4.07 2.31
#> 2 0.31 J SI2 63.3 58.0 335 4.34 4.35 2.75
#> 3 0.30 J SI1 64.0 55.0 339 4.25 4.28 2.73
#> 4 0.30 J SI1 63.4 54.0 351 4.23 4.29 2.70
#> 5 0.30 J SI1 63.8 56.0 351 4.23 4.26 2.71

# copying to a file doesn't work
tf <- tempfile(pattern = ".parquet")
copy_files(uri, tf)
#> Error: IOError: Cannot use file 'voltrondata-labs-datasets/diamonds/cut=Good/part-0.parquet' as a directory

# copying into a directory doesn't work either
tf <- tempfile()
dir.create(tf)
copy_files(uri, tf)
#> Error: IOError: Cannot use file 'voltrondata-labs-datasets/diamonds/cut=Good/part-0.parquet' as a directory
```

This works if I change everything to be directories, e.g.

```
uri <- "gs://anonymous@voltrondata-labs-datasets/diamonds/cut=Good/"
copy_files(uri, ".")
```
But, the docs for `from` and `to` state that this parameter is "A string path to a local directory or file, a URI, or a SubTreeFileSystem. Files will be copied recursively from this path."

Is this an API change or regression whereby it only now works with directories and not files, or has this only ever worked with directories and our docs are incorrect and need updating?

### Component(s)

R

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the R examples using copy_files() with the gs:// file URI and with the directory URI, then compare the documented from and to parameter behavior. Done means the implementation and documentation agree on whether individual files can be copied, with the reported error case covered by the resulting behavior or guidance.

Written by the indexing model from the issue text.

Assessment

Tech stack
google-cloud, r
Domain
api, cloud
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.