GoogleCloudPlatform / GoogleCloudPlatform/gsutil
rsync terminates with ValueError: I/O operation on closed file
- Dominant language
- Python
- Stars
- 918
- Forks
- 335
- PR merge metrics
- No merged PRs in 30d
Description
I'm running gsutil rsync in an Ubuntu 16.04 LTS Docker container. It's installed via the official command:
```
RUN export CLOUD_SDK_REPO="cloud-sdk-$(lsb_release -c -s)" && \
echo "deb http://packages.cloud.google.com/apt $CLOUD_SDK_REPO main" | tee -a /etc/apt/sources.list.d/google-cloud-sdk.list && \
curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add - && \
apt-get update -y && apt-get install google-cloud-sdk -y
```
I have a bucket with 900k files. gsutil cp -r worked in both directions. gsutil rsync however cannot do this task, it terminates after a while with:
```
Exception in thread Thread-5:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/seek_ahead_thread.py", line 97, in run
for seek_ahead_result in self.seek_ahead_iterator:
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/commands/rsync.py", line 1149, in __iter__
for diff_to_apply in self.cloned_diff_iterator:
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/commands/rsync.py", line 1057, in __iter__
self._ParseTmpFileLine(self.sorted_src_urls_it.next()))
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/plurality_checkable_iterator.py", line 65, in _PopulateHead
e = self.base_iterator.next()
ValueError: I/O operation on closed file
```
The command I've run was:
```
gsutil -m rsync -rdn /local/folder gs://bucket_name
```
Contributor guide
Assessment
This issue has not been assessed yet.