GoogleCloudPlatform / GoogleCloudPlatform/gsutil
gsutil crashes when run in hadoop's long temp directory paths
- Dominant language
- Python
- Stars
- 918
- Forks
- 335
- PR merge metrics
- No merged PRs in 30d
Description
I am attempting to use gsutil inside a hadoop streaming setup. When I try to run the following command (where I have preserved filename length but masked the names)
```
gsutil stat gs://XXXX-XX-XXXXX-XXXYXXX/XXYXXXXXYXXXXYXX
```
as my mapper I get the following error:
```
Process SyncManager-1:
Traceback (most recent call last):
File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
self._target(*self._args, **self._kwargs)
File "/usr/lib/python2.7/multiprocessing/managers.py", line 550, in _run_server
server = cls._Server(registry, address, authkey, serializer)
File "/usr/lib/python2.7/multiprocessing/managers.py", line 162, in __init__
self.listener = Listener(address=address, backlog=16)
File "/usr/lib/python2.7/multiprocessing/connection.py", line 132, in __init__
self._listener = SocketListener(address, family, backlog)
File "/usr/lib/python2.7/multiprocessing/connection.py", line 257, in __init__
self._socket.bind(address)
File "/usr/lib/python2.7/socket.py", line 224, in meth
return getattr(self._sock,name)(*args)
error: AF_UNIX path too long
```
It seems like gsutil might be creating temporary sockets in the current directory which overflow the UNIX limit when inside hadoop's long temporary paths.
My image is on `SMP Debian 3.14.15-2~bpo70+1` in case that is useful.
Contributor guide
Assessment
This issue has not been assessed yet.