GoogleCloudPlatform / GoogleCloudPlatform/gsutil
gsutil does not support retry during piped downloads
Open
Feature Request
- Dominant language
- Python
- Stars
- 918
- Forks
- 335
- PR merge metrics
- No merged PRs in 30d
Description
While uploads thankfully can retry since https://github.com/GoogleCloudPlatform/gsutil/issues/229 was fixed by @houglum, this does not work for downloads, which we see often break during streaming download of 230GiB files.
As we're essentially untarring the result, storing the intermediate file should not be necessary.
Memory buffering and re-starting a download from the place where the transfer stopped via a range request before continuing to pipe should be entirely possible.
Thank you for your consideration :)
Contributor guide
Assessment
This issue has not been assessed yet.