[C++][R] copy_files() does not take paths to specific files
- Dominant language
- C++
- Stars
- 17.1k
- Forks
- 4.3k
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 88
Description
`copy_files()` is a pretty handy function for working between local and remote interfaces, particularly for any file type arrow doesn't handle (arvo, ncdf, h5, etc etc).
Unfortunately, it seems to work only from directory-to-directory, at least in the direction of copying S3 -> local file system. e.g. this reprex:
```java
library(arrow)
local_dir <- tempfile()
fs::dir_delete(local_dir)
fs::dir_create(local_dir) # create dir if it doesn't exist
l3 <- SubTreeFileSystem$create(local_dir)
l3$ls() #empty
s3 <- s3_bucket("neon4cast-targets/aquatics", endpoint_override = "data.ecoforecast.org", anonymous=TRUE)
s3$ls() #not empty, good
copy_files(s3$path("aquatics-targets.csv.gz"), l3$path("aquatics-targets.csv.gz"))
l3$ls() # darn, nothing!
copy_files(s3$path("aquatics-targets.csv.gz"), l3)
l3$ls() # darn, nothing!
copy_files(s3, l3)
l3$ls() # Finally! only this works
```
**Reporter**: [Carl Boettiger](https://issues.apache.org/jira/browse/ARROW-17120) / @cboettig
#### Related issues:
- [[R] copy_files doesn't make it easy to copy a single file](https://github.com/apache/arrow/issues/28030) (is duplicated by)
**Note**: *This issue was originally created as [ARROW-17120](https://issues.apache.org/jira/browse/ARROW-17120). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*
Contributor guide
Research direction
The report provides an R reprex using copy_files(), SubTreeFileSystem$create(), and s3_bucket(); begin by reproducing the S3-to-local single-file cases. Done means a source and destination path for one file copies it and the result appears in l3$ls(), while directory-to-directory behavior remains intact.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, cpp, r
- Domain
- backend, cloud
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100