GoogleCloudPlatform / GoogleCloudPlatform/gsutil

gsutil with parallel_composite_upload_threshold=100M fails repeatedly to upload 30G file

Open
#294 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
918
Forks
335
PR merge metrics
No merged PRs in 30d

Description

I am trying to use GCS as a backup for my VM disks (qcow2) files. So was using the "gsutil cp" to upload the file. However, gsutil gives up after uploading only two components of the file, each of 948MB ?? Even if I try restarting the command it either keeps giving :

``` console
ResumableUploadException: 503 Server Error
ResumableUploadException: 503 Server Error
ResumableUploadException: 503 Server Error
ResumableUploadException: 503 Server Error
```

or
it show component are ready to upload but gets stuck without being able to upload a single byte:

``` console
Resuming upload for file:///home/asridharan/devsda6/qemu-kvm/stitchserver/stitchserver.qcow2
Uploading ...util_help_cp/bf5bd2a63b8821c4bc2a275fd995c1a4_15: 0 B/949.97 MiB
Uploading ...util_help_cp/bf5bd2a63b8821c4bc2a275fd995c1a4_12: 0 B/949.97 MiB
Uploading ...util_help_cp/bf5bd2a63b8821c4bc2a275fd995c1a4_17: 0 B/949.97 MiB
Uploading ...util_help_cp/bf5bd2a63b8821c4bc2a275fd995c1a4_13: 0 B/949.97 MiB
```

Is there a bandwidth limitation or storage limitation on my account that I might not be aware off?

Version I am using is:
gsutil version: 4.13

if I required I can dumb the entire log, or create a gistlog for it.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.