GoogleCloudPlatform / GoogleCloudPlatform/gsutil
gsutil hangs while syncing files to bucket
- Dominant language
- Python
- Stars
- 918
- Forks
- 335
- PR merge metrics
- No merged PRs in 30d
Description
Using the following command `gsutil -m cp -r -z html,css,js,svg /Users/***/Developer/sites/***/public_html/static/* gs://***/static/`, gsutil hangs after 62% done every time and stops on the same file. The file size of the file where it stops is 4kb.
```
Copying file:///Users/*****/Developer/sites/*******.com/public_html/static/wagtailusers/js/group-form.js [Content-Type=application/javascript]...
INFO 0415 08:49:59.231377 retry_util.py] Retrying request, attempt #1...
INFO 0415 08:50:19.189575 retry_util.py] Retrying request, attempt #2...1
INFO 0415 08:50:41.422077 retry_util.py] Retrying request, attempt #3...
INFO 0415 08:51:06.343943 retry_util.py] Retrying request, attempt #4...
INFO 0415 08:51:42.249578 retry_util.py] Retrying request, attempt #5...
INFO 0415 08:52:32.441719 retry_util.py] Retrying request, attempt #6...
INFO 0415 08:53:22.623934 retry_util.py] Retrying request, attempt #7...
INFO 0415 08:54:12.976616 retry_util.py] Retrying request, attempt #8...
INFO 0415 08:55:03.169330 retry_util.py] Retrying request, attempt #9...
INFO 0415 08:55:55.434091 retry_util.py] Retrying request, attempt #10...
INFO 0415 08:56:46.884525 retry_util.py] Retrying request, attempt #11...
INFO 0415 08:57:38.080628 retry_util.py] Retrying request, attempt #12...
INFO 0415 08:58:29.483614 retry_util.py] Retrying request, attempt #13...
INFO 0415 08:59:20.170236 retry_util.py] Retrying request, attempt #14...
\ [708/719 files][ 23.7 MiB/ 37.9 MiB] 62% Done 0.0 B/s
```
Is there any way to get more information about what went wrong?
Removing `-z html,css,js,svg` and adding `-o "GSUtil:parallel_process_count=1"` makes it work. Why?
` gsutil -m -o "GSUtil:parallel_process_count=1" cp -r /Users/***/Developer/sites/***/public_html/static/* gs://***/static/`
gsutil version: 4.61
Cloud SDK version: 336.0.0
Contributor guide
Research direction
Start by reproducing the exact gsutil 4.61 command with and without -z and with parallel_process_count=1. Inspect the retry_util.py retry output and compare the parallel upload paths around the file named in the log. Done means identifying why the parallel compressed sync stalls and documenting or fixing the failure with a regression test or clearer diagnostics.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- gcp, python
- Domain
- cli, cloud
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 28/100