GoogleCloudPlatform / GoogleCloudPlatform/gsutil
Connection error when emptying bucket (using -m)
- Dominant language
- Python
- Stars
- 918
- Forks
- 335
- PR merge metrics
- No merged PRs in 30d
Description
Although I cannot consistently reproduce this issue, running `gsutil -m rm -r gs://${BUCKET}/ ` resulted in:
```sh
Exception in thread Thread-5:
Traceback (most recent call last):
File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/managers.py", line 811, in _callmethod
conn = self._tls.connection
AttributeError: 'ForkAwareLocal' object has no attribute 'connection'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/local/bin/google-cloud-sdk/platform/gsutil/gslib/command.py", line 2348, in run
cls = copy.copy(class_map[caller_id])
File "", line 2, in __getitem__
File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/managers.py", line 815, in _callmethod
self._connect()
File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/managers.py", line 802, in _connect
conn = self._Client(self._token.address, authkey=self._authkey)
File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/connection.py", line 492, in Client
c = SocketClient(address)
File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/connection.py", line 619, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 61] Connection refused
```
The bucket had approximately 300 objects, and eventually all but one removed -- one object was left dangling. This error did _not_ block the removal process from running, but I figured I would report it either way.
System:
- macOS 10.15.5 Catalina, 2.6 GHz 6-Core Intel Core i7
- gsutil version: 4.51
Contributor guide
Research direction
Start with gslib/command.py around line 2348 and the multiprocessing manager traceback shown in the report. Run gsutil -m rm -r against a bucket with many objects, if a reproducible setup is available, and inspect the worker failure and the dangling object. Done means the command completes without the connection-refused traceback and removes all targeted objects.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cli
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100