saltstack / saltstack/salt

[BUG] `salt-call --local -g --out yaml` redirected stdoutput prints bytestring on Simplified Chinese Windows

Open
#62,399 3 comments 0 reactions 1 assignee View on GitHub

@dwoz is already working on this.

Since Aug 2, 2022.

bug bytes-vs-strings needs-triage Windows
Dominant language
Python
Stars
15.7k
Forks
5.6k
Avg merge
2d 44m
Merged PRs (30d)
80

Description

Description
Given a Windows with the locale on Simplified Chinese, or Taiwanese (they have different cp so fixing one might not fix the other)
If you run salt-call --local grains.items --out yml (reproduces without --out yml or with --out json) and redirect the output (programatically or using the > file.txt in command-line/powershell) the output is a binary-string as a string.

Example:
image

The problem we're facing is that we're using salt-call grains.items --out yml from a C# Application, redirect the Process stdout, and then we try to deserialize the values using an YamlDotNet.
The deserialization fails because we're getting the b"local:..." (with b" as the first characters and \n as escaped characters) as a string so it's not a valid YAML.

Important findings
If you go to salt/utils/yamldumper.py and change the line 127 to kwargs["allow_unicode"] = False

  • If you run salt-call --local -g --out yml now the output is correct
  • If you run salt-call --local -g (without --out yml) the output remains the same

Setup

  • Installed a Hyper-V Windows 11 machine (works also with Windows 10)
  • Installed the Simplified Chinese or the Taiwanese language pack and set that to the Windows Language from the Region Settings
  • Installed salt 3004.4 from the repo https://archive.repo.saltproject.io/windows/

This has been reproduced both on an actual machine and a VM.

  • on-prem machine
  • VM (Virtualbox, KVM, etc. please specify) Hyper-V
  • VM running on a cloud service, please be explicit and add details
  • container (Kubernetes, Docker, containerd, etc. please specify)
  • or a combination, please be explicit
  • jails if it is FreeBSD

Steps to Reproduce the behavior
Easiest way to reproduce it:

  • Run salt-call --local -g --out yaml > output.txt from cmd or powershell.
  • output.txt now contains a string of the format b'local: etc.. which is not a valid yaml.

This has been reproduced on many salt versions, some have different outputs than others:

  • 3000.1, 3001, 3001.1 -> the string starts with b" (it starts with double-quotes)
  • 3004, 3004.4 -> the string starts with b' (it starts with single-quote)

Expected behavior

  • output.txt should contain a string of the format (valid yaml)
local:
  ...

Screenshots
image

Versions Report

salt --versions-report (Provided by running salt --versions-report. Please also mention any differences in master/minion versions.)
Salt Version:
          Salt: 3004
 
Dependency Versions:
          cffi: 1.14.6
      cherrypy: 18.6.1
      dateutil: 2.8.1
     docker-py: Not Installed
         gitdb: 4.0.7
     gitpython: 3.1.18
        Jinja2: 2.10.1
       libgit2: Not Installed
      M2Crypto: Not Installed
          Mako: 1.1.4
       msgpack: 0.6.2
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     pycparser: 2.20
      pycrypto: Not Installed
  pycryptodome: 3.10.1
        pygit2: Not Installed
        Python: 3.8.8 (tags/v3.8.8:024d805, Feb 19 2021, 13:18:16) [MSC v.1928 64 bit (AMD64)]
  python-gnupg: 0.4.7
        PyYAML: 5.4.1
         PyZMQ: 19.0.0
         smmap: 4.0.0
       timelib: 0.2.4
       Tornado: 4.5.3
           ZMQ: 4.3.2
 
System Versions:
          dist:   
        locale: cp1252
       machine: AMD64
       release: 10
        system: Windows
       version: 10 10.0.22000 SP0 Multiprocessor Free

Additional context
These are all my findings..

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.