GoogleCloudPlatform / GoogleCloudPlatform/gsutil
Windows in German: gsutil cp -r throws UnicodeDecodeError when bucket name contains `:`
- Dominant language
- Python
- Stars
- 918
- Forks
- 335
- PR merge metrics
- No merged PRs in 30d
Description
I followed the instructions on how to export Firestore data into a G Cloud Bucket and to clone it onto my local machine. Because of a mix of a file system naming issue and a localized Windows error message, I got an `UnicodeDecodeError`.
My steps:
- I created a bucket and exported data from Firebase's Firestore into it with `gcloud firestore export gs://todo-james-testing-data`
- The tool automatically created the bucket name `2020-05-07T00:50:58_99460` and copied the data there
- I ran `gsutil cp -r gs://my-bucket-name .` on my local machine
- I got the error message `UnicodeDecodeError: 'ascii' codec can't decode byte 0xfc in position 12: ordinal not in range(128)` (full error below)
- After tracking down the problem in the stack trace and removing `except`s, I got the actual error `WindowsError: [Error 123] Die Syntax f³r den Dateinamen, Verzeichnisnamen oder die Datentrõgerbezeichnung ist falsch: u'.\\my-bucket-name\\2020-05-07T00:50:58_99460'`
- Apparently, Windows had thrown an error message in German containing umlauts (ü in für and ä in Datenträgerbezeichnung). One of the `except` clauses in `C:\Users\Paul\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\gsutil\gslib\__main__.py` catched that and tried to show the error message to me, failing because of the umlauts.
- It seems that the Windows error came up in the first place because on Windows, `:` is not allowed in a filename. Because of this, the auto-generated name failed to download.
- I fixed the problem for me by changing the bucket name to something without a `:`. However, I think a more expressive error message than `UnicodeDecodeError` for an underlying file system error would be nice.
Full stack trace:
```Copying gs://todo-james-testing-data/2020-05-07T00:50:58_99460/2020-05-07T00:50:58_99460.overall_export_metadata...
Traceback (most recent call last):
File "C:\Users\Paul\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform/gsutil\gsutil", line 21, in
gsutil.RunMain()
File "C:\Users\Paul\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\gsutil\gsutil.py", line 124, in RunMain
sys.exit(gslib.__main__.main())
File "C:\Users\Paul\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\gsutil\gslib\__main__.py", line 433, in main
user_project=user_project)
File "C:\Users\Paul\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\gsutil\gslib\__main__.py", line 659, in _RunNamedCommandAndHandleExceptions
_OutputAndExit(message='OSError: %s.' % e.strerror, exception=e)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xfc in position 12: ordinal not in range(128)```
Contributor guide
Assessment
This issue has not been assessed yet.