saltstack / saltstack/salt

[BUG] salt-cp UnicodeDecodeError: 'utf-8'

Open
#59,899 4 comments 2 reactions 2 assignees View on GitHub

@xeacott is already working on this.

Since Mar 25, 2021.

bug Confirmed severity-medium Windows
Dominant language
Python
Stars
15.7k
Forks
5.6k
Avg merge
2d 44m
Merged PRs (30d)
80

Description

Description of Issue

Copying windows files produces UnicodeDecodeError: 'utf-8' codec can't decode byte 0xba in position 10: invalid start byte

ERROR   ] An un-handled exception was caught by salt's global exception handler:
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xba in position 10: invalid start byte
Traceback (most recent call last):
  File "/usr/bin/salt-cp", line 11, in <module>
    load_entry_point('salt==3002.5', 'console_scripts', 'salt-cp')()
  File "/usr/lib/python3/dist-packages/salt/scripts.py", line 429, in salt_cp
    client.run()
  File "/usr/lib/python3/dist-packages/salt/cli/cp.py", line 53, in run
    cp_.run()
  File "/usr/lib/python3/dist-packages/salt/cli/cp.py", line 145, in run
    ret = self.run_oldstyle()
  File "/usr/lib/python3/dist-packages/salt/cli/cp.py", line 153, in run_oldstyle
    arg = [self._load_files(), self.opts["dest"]]
  File "/usr/lib/python3/dist-packages/salt/cli/cp.py", line 128, in _load_files
    files.update(self._file_dict(fn_))
  File "/usr/lib/python3/dist-packages/salt/cli/cp.py", line 117, in _file_dict
    data = fp_.read()
  File "/usr/lib/python3.5/codecs.py", line 321, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xba in position 10: invalid start byte
Traceback (most recent call last):
  File "/usr/bin/salt-cp", line 11, in <module>
    load_entry_point('salt==3002.5', 'console_scripts', 'salt-cp')()
  File "/usr/lib/python3/dist-packages/salt/scripts.py", line 429, in salt_cp
    client.run()
  File "/usr/lib/python3/dist-packages/salt/cli/cp.py", line 53, in run
    cp_.run()
  File "/usr/lib/python3/dist-packages/salt/cli/cp.py", line 145, in run
    ret = self.run_oldstyle()
  File "/usr/lib/python3/dist-packages/salt/cli/cp.py", line 153, in run_oldstyle
    arg = [self._load_files(), self.opts["dest"]]
  File "/usr/lib/python3/dist-packages/salt/cli/cp.py", line 128, in _load_files
    files.update(self._file_dict(fn_))
  File "/usr/lib/python3/dist-packages/salt/cli/cp.py", line 117, in _file_dict
    data = fp_.read()
  File "/usr/lib/python3.5/codecs.py", line 321, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xba in position 10: invalid start byte

Copying files created on linux machine working fine.

Setup

master:

Salt Version:
          Salt: 3002.5
 
Dependency Versions:
          cffi: 1.13.2
      cherrypy: 3.5.0
      dateutil: 2.5.3
     docker-py: 1.9.0
         gitdb: 2.0.0
     gitpython: 2.1.1
        Jinja2: 2.9.4
       libgit2: 0.28.4
      M2Crypto: Not Installed
          Mako: Not Installed
       msgpack: 0.6.2
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     pycparser: 2.19
      pycrypto: 2.6.1
  pycryptodome: 3.6.1
        pygit2: 1.0.2
        Python: 3.5.3 (default, Nov 18 2020, 21:09:16)
  python-gnupg: Not Installed
        PyYAML: 3.12
         PyZMQ: 17.1.2
         smmap: 2.0.1
       timelib: Not Installed
       Tornado: 4.5.3
           ZMQ: 4.2.1
 
System Versions:
          dist: debian 9 stretch
        locale: UTF-8
       machine: x86_64
       release: 4.9.0-14-amd64
        system: Linux
       version: Debian GNU/Linux 9 stretch

Minion:
`PS C:\salt> ./salt-minion --versions
Salt Version:
Salt: 3001.4

Dependency Versions:
cffi: 1.12.2
cherrypy: 17.4.1
dateutil: 2.8.0
docker-py: Not Installed
gitdb: 2.0.5
gitpython: Not Installed
Jinja2: 2.10.1
libgit2: Not Installed
M2Crypto: Not Installed
Mako: 1.0.7
msgpack-pure: Not Installed
msgpack-python: 1.0.0
mysql-python: Not Installed
pycparser: 2.19
pycrypto: Not Installed
pycryptodome: 3.9.7
pygit2: Not Installed
Python: 3.7.4 (tags/v3.7.4:e09359112e, Jul 8 2019, 20:34:20) [MSC v.1916 64 bit (AMD64)]
python-gnupg: 0.4.4
PyYAML: 5.3.1
PyZMQ: 18.0.1
smmap: 2.0.5
timelib: 0.2.4
Tornado: 4.5.3
ZMQ: 4.3.1

System Versions:
dist:
locale: cp1250
machine: AMD64
release: 2016Server
system: Windows
version: 2016Server 10.0.14393 SP0`

Steps to Reproduce Issue

download file from windows in my case it's https://www.catalog.update.microsoft.com/Search.aspx?q=KB4340355
copy files to minion using:
salt-cp 'server' sqlserver2016kb4340355.exe "C:\\scripts\\sqlserver2016kb4340355.exe"

Workaround:

When i use --chunked file is successfully copied to windows but copying is very slow :(
salt-cp -C 'server' sqlserver2016kb4340355.exe "C:\scripts\sqlserver2016kb4340355.exe"

I tried to make a zip file at linux machine after download this file but it also throws errors

Versions Report

Master: 3002.5 @ debian 9 stretch
Minion: 3001.4 @ 2016Server 10.0.14393 SP0

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.