apache / apache/arrow-rs-object-store
Object_store: put_file and get_file methods
- Dominant language
- Rust
- Stars
- 322
- Forks
- 212
- Avg merge
- 5d 2h
- Merged PRs (30d)
- 10
Description
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
As a user, I would like to utilize the object_store to put and retrieve files. Currently, I can employ the put and get methods for this purpose. However, I am responsible for handling any issues that arise. For instance, when uploading a file to the store, I must verify its size to determine whether multipart upload is necessary. Similarly, when retrieving a large file, I cannot download it in its entirety before writing it to the local file system, as this would cause excessive memory usage. Instead, I may need to use get_range and write specific portions of the file locally. Moreover, downloading different ranges could potentially be executed in parallel.
**Describe the solution you'd like**
Expose get_file(local_path, remote_path) and put_file(local_path, remote_path) to be more user-friendly.
**Describe alternatives you've considered**
Stay on what we have now, which means users should implement the functionalities by themself.
Contributor guide
Assessment
This issue has not been assessed yet.