googleapis / googleapis/google-cloud-python

Allow transfer_manager to upload/download file-like object

Open
#15,971 1 comment 1 reaction 1 assignee Assigned to @chandra-siri View on GitHub
api: storage type: feature request
Dominant language
Python
Stars
5.4k
Forks
1.8k
Avg merge
3d 4h
Merged PRs (30d)
122

Description

**Is your feature request related to a problem? Please describe.**
We'd like to have `download_chunks_concurrently` and `upload_chunks_concurrently` support file-like objects. That way, we can upload the model checkpoints directly from memory instead of writing them to disk first.

**Describe the solution you'd like**
Support passing a file-like object, such as `BytesIO`, to the `filename` in the function's argument.

**Describe alternatives you've considered**
Due to the lack of this functionality, I need to write my data in a local file and then upload the file to object storage.

**Additional context**
N/A

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.