saltstack / saltstack/salt

UnicodeEncodeError when applying states from master but not minion

Open
#51,496 13 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug severity-medium
Dominant language
Python
Stars
15.7k
Forks
5.6k
Avg merge
2d 44m
Merged PRs (30d)
80

Description

Description of Issue/Question

When running state.apply from the master, I get this:

          ID: Suran VCS_localizable_CORE_RES_1
    Function: file.replace
        Name: /Users/Shared/Jenkins/Applications/Omnis/master/Suran VCS.app/Contents/Resources/English.lproj/Localizable.strings
      Result: False
     Comment: An exception occurred in this state: Traceback (most recent call last):
                File "/opt/salt/lib/python3.5/site-packages/salt/state.py", line 1913, in call
                  **cdata['kwargs'])
                File "/opt/salt/lib/python3.5/site-packages/salt/loader.py", line 1898, in wrapper
                  return f(*args, **kwargs)
                File "/opt/salt/lib/python3.5/site-packages/salt/states/file.py", line 4301, in replace
                  backslash_literal=backslash_literal)
                File "/opt/salt/lib/python3.5/site-packages/salt/modules/file.py", line 2341, in replace
                  w_file.write(salt.utils.stringutils.to_str(result))
              UnicodeEncodeError: 'ascii' codec can't encode character '\xa9' in position 14052: ordinal not in range(128)
     Started: 15:34:39.314528
    Duration: 1120.305 ms
     Changes:

But, when I run salt-call state.apply from the minion, the state applies properly:

          ID: Suran VCS_localizable_CORE_RES_1
    Function: file.replace
        Name: /Users/Shared/Jenkins/Applications/Omnis/master/Suran VCS.app/Contents/Resources/English.lproj/Localizable.strings
      Result: True
     Comment: Changes were made
     Started: 15:38:21.728764
    Duration: 253.199 ms
     Changes:
              ----------
              diff:
                  ---
                  +++
                  @@ -1,5 +1,5 @@
                   {
                  -"CORE_RES_1" = "Omnis Studio";
                  +"CORE_RES_1" = "Suran VCS";
                   "CORE_RES_2" = "Version 8.1.6";
                   "CORE_RES_3" = "Omnis Studio $ $";
                   "CORE_RES_4" = "";
Setup
Suran VCS_localizable_CORE_RES_1:
  file.replace:
    - name: /Users/Shared/Jenkins/Applications/Omnis/master/Suran VCS.app/Contents/Resources/English.lproj/Localizable.strings
    - pattern: |
        "CORE_RES_1" = ".*";
    - repl: |
        "CORE_RES_1" = "Suran VCS";
    - require:
      - file: /Users/Shared/Jenkins/Applications/Omnis/master/Suran VCS.app

Sample Localizable.string file

Versions Report

Minion:

Salt Version:
           Salt: 2018.3.3

Dependency Versions:
           cffi: 1.11.2
       cherrypy: unknown
       dateutil: 2.6.1
      docker-py: Not Installed
          gitdb: 2.0.5
      gitpython: 2.1.7
          ioflo: Not Installed
         Jinja2: 2.10
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: 1.0.7
   msgpack-pure: Not Installed
 msgpack-python: 0.4.8
   mysql-python: Not Installed
      pycparser: 2.18
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 3.5.3 (default, Nov  8 2018, 15:57:22)
   python-gnupg: 0.4.1
         PyYAML: 3.12
          PyZMQ: 17.0.0
           RAET: Not Installed
          smmap: 2.0.5
        timelib: 0.2.4
        Tornado: 4.5.2
            ZMQ: 4.1.6

System Versions:
           dist:
         locale: UTF-8
        machine: x86_64
        release: 15.6.0
         system: Darwin
        version: 10.11.6 x86_64

Master:

Salt Version:
           Salt: 2018.3.3

Dependency Versions:
           cffi: 1.5.2
       cherrypy: Not Installed
       dateutil: Not Installed
      docker-py: Not Installed
          gitdb: 0.6.4
      gitpython: 1.0.1
          ioflo: Not Installed
         Jinja2: 2.8
        libgit2: 0.24.0
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.4.6
   mysql-python: Not Installed
      pycparser: 2.14
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: 0.24.0
         Python: 3.5.2 (default, Nov 12 2018, 13:43:14)
   python-gnupg: Not Installed
         PyYAML: 3.11
          PyZMQ: 15.2.0
           RAET: Not Installed
          smmap: 0.9.0
        timelib: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.1.4

System Versions:
           dist: Ubuntu 16.04 xenial
         locale: UTF-8
        machine: x86_64
        release: 3.13.0-57-generic
         system: Linux
        version: Ubuntu 16.04 xenial

I'm at a loss to explain why salt behaves differently based on who initiates the state apply.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with salt/modules/file.py around the replace implementation and salt/states/file.py around the reported call, then inspect salt.utils.stringutils.to_str. Compare the master-initiated and minion-local execution paths using the provided state and Unicode-containing Localizable.strings file. Done means the same state applies successfully from both paths without the UnicodeEncodeError, with regression coverage for the differing environments.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
devops, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.