GoogleCloudPlatform / GoogleCloudPlatform/gsutil

Crash in 'gsutil help metadata'

Open
#932 1 comment 2 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
918
Forks
335
PR merge metrics
No merged PRs in 30d

Description

'gsutil help metadata' crashes consistently with the error

> UnicodeEncodeError: 'ascii' codec can't encode character u'\xe3' in position 7114: ordinal not in range(128)

`gsutil help` for several other topics works. This is the only one that I found to make it go boom.

First spotted in v4.45; upgrade to 4.46 did not change the behavior. gsutil is installed as part of the Google Cloud SDK.

The bug is far from a show-stopper, but would be nice to have fixed.

Stack trace
```
$ gsutil help metadata
Traceback (most recent call last):
File "/usr/lib/google-cloud-sdk/platform/gsutil/gsutil", line 21, in
gsutil.RunMain()
File "/usr/lib/google-cloud-sdk/platform/gsutil/gsutil.py", line 124, in RunMain
sys.exit(gslib.__main__.main())
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/__main__.py", line 433, in main
user_project=user_project)
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/__main__.py", line 762, in _RunNamedCommandAndHandleExceptions
_HandleUnknownFailure(e)
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/__main__.py", line 628, in _RunNamedCommandAndHandleExceptions
user_project=user_project)
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/command_runner.py", line 411, in RunNamedCommand
return_code = command_inst.RunCommand()
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/commands/help.py", line 173, in RunCommand
self._OutputHelp(''.join(output))
File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/commands/help.py", line 202, in _OutputHelp
universal_newlines=True).communicate(input=help_str)
File "/usr/lib/python2.7/subprocess.py", line 469, in communicate
self.stdin.write(input)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe3' in position 7114: ordinal not in range(128)
```
System/environment information
```
$ gsutil --version
gsutil version: 4.46
$ python2.7 --version
Python 2.7.16
$ locale
LANG=C.UTF-8
LANGUAGE=
LC_CTYPE="C.UTF-8"
LC_NUMERIC="C.UTF-8"
LC_TIME="C.UTF-8"
LC_COLLATE="C.UTF-8"
LC_MONETARY="C.UTF-8"
LC_MESSAGES="C.UTF-8"
LC_PAPER="C.UTF-8"
LC_NAME="C.UTF-8"
LC_ADDRESS="C.UTF-8"
LC_TELEPHONE="C.UTF-8"
LC_MEASUREMENT="C.UTF-8"
LC_IDENTIFICATION="C.UTF-8"
LC_ALL=
$ uname -srvmo
Linux 4.19.0-6-amd64 #1 SMP Debian 4.19.67-2+deb10u1 (2019-09-20) x86_64 GNU/Linux
$ lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description: Debian GNU/Linux 10 (buster)
Release: 10
Codename: buster
```
---
**ADDITIONAL INFO** discovered since I reported it couple weeks ago.

`CLOUDSDK_GSUTIL_PYTHON=python3 gsutil help metadata` works on my work machine, the same I reported the issue against (by default, python 2.7 is used, as seen in the stack trace).

The command works "out of the box" in the Cloud devshell. Whatever I tried, I could not make it fail there. First make sure locale settings are exactly same as I have at home:
```
[snip]@cloudshell:~$ LC_ALL=C.UTF-8 LANG=C.UTF-8 locale
LANG=C.UTF-8
LANGUAGE=
LC_CTYPE="C.UTF-8"
LC_NUMERIC="C.UTF-8"
LC_TIME="C.UTF-8"
LC_COLLATE="C.UTF-8"
LC_MONETARY="C.UTF-8"
LC_MESSAGES="C.UTF-8"
LC_PAPER="C.UTF-8"
LC_NAME="C.UTF-8"
LC_ADDRESS="C.UTF-8"
LC_TELEPHONE="C.UTF-8"
LC_MEASUREMENT="C.UTF-8"
LC_IDENTIFICATION="C.UTF-8"
LC_ALL=C.UTF-8
```
And still the command
```
[snip]@cloudshell:~$ CLOUDSDK_GSUTIL_PYTHON=python2.7 LC_ALL=C.UTF-8 LANG=C.UTF-8 gsutil help metadata
```
works just fine. Just to paranoidally double-check, the exactly same command with environment settings throws the same originally reported exception on my box. Gsutil version is 4.46 on both. Looks like a tough one to repro, sorry. I wish my report could be more helpful.

The only difference I'm spotting is devshell is based on Debian 9 and has python 2.7.13, while my machine, where it fails, is Debian 10 with python 2.7.16. But, if I'm to trust my SWE intuition, it's unlikely where the root cause is.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.