GoogleCloudPlatform / GoogleCloudPlatform/gsutil
gsutil rsync hangs on file transfer
- Dominant language
- Python
- Stars
- 918
- Forks
- 335
- PR merge metrics
- No merged PRs in 30d
Description
I am using gsutil rsync to sync directories from a regular POSIX fileserver over to GCP buckets.
One on particular directory, gsutil rsync hangs. I'd like to figure out what it is that is making it hang. I tried running it under strace, and also with the "-D" debug mode.
example command:
/root/google-cloud-sdk/bin/gsutil -D -m rsync -C -e -P -r -x '(.gnupg|.cache|.*Icon\r$)' /home01/whatever gs://bucket-name
Here is the output when it gets stuck:
crc32c: u'n3lv8A=='
generation: 1515639588224581
md5Hash: u'8Ci6mkl3FqcYhC2GMbfzvQ=='
metadata: , , , , ]>
name: u'test.txt'
size: 25
timeCreated: datetime.datetime(2018, 1, 11, 2, 59, 48, 172000, tzinfo=)>]
prefixes: []>
At source listing 300000...
Starting synchronization
process count: 20
thread count: 5
/ [0/1 files][ 0.0 B/ 0.0 B]
So there is some 1 file somewhere that it somehow gets stuck on. How can I troubleshoot further? I tried stracing my command and looking for all open and access calls, hoping to hit upon the problematic file.
At some point I noticed a file that had mode 000, so I thought that was the problem, but I was not able to reproduce that, that is, making a new file with mode 000 uploads it just fine.
OK, I tried the same thing without "-m" but with "-D" and I get:
...
acl: []
crc32c: u'n3lv8A=='
generation: 1515639588224581
md5Hash: u'8Ci6mkl3FqcYhC2GMbfzvQ=='
metadata: , , , , ]>
name: u'test.txt'
size: 25
timeCreated: datetime.datetime(2018, 1, 11, 2, 59, 48, 172000, tzinfo=)>]
prefixes: []>
Starting synchronization
process count: 1
thread count: 1
and then the rsync hangs there indefinitely.
Contributor guide
Assessment
This issue has not been assessed yet.