GoogleCloudPlatform / GoogleCloudPlatform/gsutil

rsync terminates with ValueError: I/O operation on closed file

Open
#574 3 comments 3 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
918
Forks
335
PR merge metrics
No merged PRs in 30d

Description

I'm running gsutil rsync in an Ubuntu 16.04 LTS Docker container. It's installed via the official command:

```
RUN export CLOUD_SDK_REPO="cloud-sdk-$(lsb_release -c -s)" && \
echo "deb http://packages.cloud.google.com/apt $CLOUD_SDK_REPO main" | tee -a /etc/apt/sources.list.d/google-cloud-sdk.list && \
curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add - && \
apt-get update -y && apt-get install google-cloud-sdk -y
```

I have a bucket with 900k files. gsutil cp -r worked in both directions. gsutil rsync however cannot do this task, it terminates after a while with:

```
Exception in thread Thread-5:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/seek_ahead_thread.py", line 97, in run
for seek_ahead_result in self.seek_ahead_iterator:
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/commands/rsync.py", line 1149, in __iter__
for diff_to_apply in self.cloned_diff_iterator:
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/commands/rsync.py", line 1057, in __iter__
self._ParseTmpFileLine(self.sorted_src_urls_it.next()))
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/plurality_checkable_iterator.py", line 65, in _PopulateHead
e = self.base_iterator.next()
ValueError: I/O operation on closed file
```

The command I've run was:
```
gsutil -m rsync -rdn /local/folder gs://bucket_name
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.