googleapis / googleapis/google-cloud-python
Allow transfer_manager to upload/download file-like object
Open
api: storage
type: feature request
- Dominant language
- Python
- Stars
- 5.4k
- Forks
- 1.8k
- Avg merge
- 3d 4h
- Merged PRs (30d)
- 122
Description
**Is your feature request related to a problem? Please describe.**
We'd like to have `download_chunks_concurrently` and `upload_chunks_concurrently` support file-like objects. That way, we can upload the model checkpoints directly from memory instead of writing them to disk first.
**Describe the solution you'd like**
Support passing a file-like object, such as `BytesIO`, to the `filename` in the function's argument.
**Describe alternatives you've considered**
Due to the lack of this functionality, I need to write my data in a local file and then upload the file to object storage.
**Additional context**
N/A
Contributor guide
Assessment
This issue has not been assessed yet.