2019.2 External file_tree pillar is broken with salt-ssh
Open
Nobody has claimed this yet.
bug
Salt-SSH
severity-high
- Dominant language
- Python
- Stars
- 15.7k
- Forks
- 5.6k
- Avg merge
- 2d 44m
- Merged PRs (30d)
- 80
Description
Description of Issue
Having configured a file tree external pillar (without templating enabled), salt-ssh call results in the following trace:
[ERROR ] TypeError encountered executing state.apply: Object of type bytes is not JSON serializable
Traceback (most recent call last):
File "/Users/user1/Library/Python/3.7/lib/python/site-packages/salt/client/ssh/__init__.py", line 1159, in run_wfunc
result = self.wfuncs[self.fun](*self.args, **self.kwargs)
File "/Users/user1/Library/Python/3.7/lib/python/site-packages/salt/client/ssh/wrapper/state.py", line 513, in apply_
return highstate(**kwargs)
File "/Users/user1/Library/Python/3.7/lib/python/site-packages/salt/client/ssh/wrapper/state.py", line 705, in highstate
roster_grains)
File "/Users/user1/Library/Python/3.7/lib/python/site-packages/salt/client/ssh/state.py", line 193, in prep_trans_tar
salt.utils.json.dump(pillar, fp_)
File "/Users/user1/Library/Python/3.7/lib/python/site-packages/salt/utils/json.py", line 121, in dump
return json_module.dump(obj, fp, **kwargs) # future lint: blacklisted-function
File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/__init__.py", line 179, in dump
for chunk in iterable:
File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/encoder.py", line 431, in _iterencode
yield from _iterencode_dict(o, _current_indent_level)
File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/encoder.py", line 405, in _iterencode_dict
yield from chunks
File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/encoder.py", line 405, in _iterencode_dict
yield from chunks
File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/encoder.py", line 438, in _iterencode
o = _default(o)
File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/encoder.py", line 179, in default
raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type bytes is not JSON serializable
This does not occur when using Python2.7.
Setup
Setup pillar.file_tree
ext_pillar:
- file_tree:
root_dir: file-tree
follow_dir_links: False
keep_newline: True
Place binary file in file_tree.
Steps to Reproduce Issue
Attempt to apply state using salt-ssh.
Versions Report
Salt Version:
Salt: 2019.2.0
Dependency Versions:
cffi: Not Installed
cherrypy: Not Installed
dateutil: 2.7.3
docker-py: Not Installed
gitdb: Not Installed
gitpython: Not Installed
ioflo: Not Installed
Jinja2: 2.10.1
libgit2: Not Installed
libnacl: Not Installed
M2Crypto: Not Installed
Mako: 1.0.12
msgpack-pure: Not Installed
msgpack-python: 0.6.1
mysql-python: Not Installed
pycparser: Not Installed
pycrypto: 2.6.1
pycryptodome: Not Installed
pygit2: Not Installed
Python: 3.7.3 (default, Mar 27 2019, 09:23:15)
python-gnupg: Not Installed
PyYAML: 3.13
PyZMQ: 18.0.1
RAET: Not Installed
smmap: Not Installed
timelib: Not Installed
Tornado: 4.5.3
ZMQ: 4.3.1
System Versions:
dist:
locale: UTF-8
machine: x86_64
release: 18.6.0
system: Darwin
version: 10.14.5 x86_64
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in salt/client/ssh/state.py at prep_trans_tar, where the pillar is written with salt.utils.json.dump, and review salt/utils/json.py. Reproduce the issue with a file_tree pillar containing a binary file and a salt-ssh state.apply call under Python 3. Done means the pillar can be transferred without the bytes-not-JSON-serializable error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cli
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100