GoogleCloudPlatform / GoogleCloudPlatform/gsutil

gsutil cp hangs for files over 1MB when transfering to s3 bucket

Open
#1,201 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
918
Forks
335
PR merge metrics
No merged PRs in 30d

Description

running the following on a google compute VM just hangs for files over 1MB. No issues with small files (30kb).
```
gsutil cp gs://bucket/subfolder/file s3://bucket/
```
Gsutil version 4.59

The same issue is seen on my local machine running version 4.53

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the `gsutil cp gs://bucket/subfolder/file s3://bucket/` command with a file larger than 1MB, using the reported gsutil versions and both the Google Compute VM and local environment where possible. Trace the `gsutil cp` entry point and compare it with the successful 30KB transfer. Done means cross-cloud copies over 1MB complete instead of hanging, with regression coverage for the reported case.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, python
Domain
cli, cloud
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.