saltstack / saltstack/salt

[BUG] salt-ssh state files do not render when using stateconf renderer due to TypeError

Open
#62,130 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug needs-triage
Dominant language
Python
Stars
15.7k
Forks
5.6k
Avg merge
2d 44m
Merged PRs (30d)
80

Description

Description
We're attempting to upgrade to v3004.1 from v2019.2, and any of our state files written with #!stateconf or #!stateconf -o are causing the stateconf renderer to throw a critical error when trying to render states to be applied via Salt-SSH using v3004.1. The same states apply correctly via Minion using v2019.2, but even a trivial state fails to apply using v3004.1 and Salt-SSH if it is using stateconf:

Trivial state file:

#!stateconf -o

.fuzfile:
  file.absent:
    - name: /tmp/myfuzfile

.sleeper:
  cmd.run:
    - name: sleep 180

Debug output:

[DEBUG   ] Returning file list from cache: age=6 cache_time=20 /tmp/tmp.wRuVuZc3Ou/cache/file_lists/roots/base.p
[DEBUG   ] Could not find file 'salt://fuz.sls' in saltenv 'base'
[DEBUG   ] In saltenv 'base', looking at rel_path 'fuz/init.sls' to resolve 'salt://fuz/init.sls'
[DEBUG   ] In saltenv 'base', ** considering ** path '/var/tmp/.f7fd55/running_data/var/cache/salt/minion/files/base/fuz/init.sls' to resolve 'salt://fuz/init.sls'
[DEBUG   ] Fetching file from saltenv 'base', ** attempting ** 'salt://fuz/init.sls'
[DEBUG   ] No dest file found
[INFO    ] Fetching file from saltenv 'base', ** done ** 'fuz/init.sls'
[DEBUG   ] compile template: /var/tmp/.f7fd55/running_data/var/cache/salt/minion/files/base/fuz/init.sls
[DEBUG   ] LazyLoaded stateconf.render
[CRITICAL] Rendering SLS fuz failed, render error: LoadedFunc object got multiple values for keyword argument 'context'
Traceback (most recent call last):
  File "/home/<user redacted>/.salt-ssh/lib/python3.6/site-packages/salt/state.py", line 4034, in render_state
    context=context,
  File "/home/<user redacted>/.salt-ssh/lib/python3.6/site-packages/salt/template.py", line 99, in compile_template
    ret = render(input_data, saltenv, sls, **render_kwargs)
  File "/home/<user redacted>/.salt-ssh/lib/python3.6/site-packages/salt/loader/lazy.py", line 149, in __call__
    return self.loader.run(run_func, *args, **kwargs)
  File "/home/<user redacted>/.salt-ssh/lib/python3.6/site-packages/salt/loader/lazy.py", line 1201, in run
    return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs)
  File "/home/<user redacted>/.salt-ssh/lib/python3.6/site-packages/contextvars/__init__.py", line 38, in run
    return callable(*args, **kwargs)
  File "/home/<user redacted>/.salt-ssh/lib/python3.6/site-packages/salt/loader/lazy.py", line 1216, in _run_as
    return _func_or_method(*args, **kwargs)
  File "/home/<user redacted>/.salt-ssh/lib/python3.6/site-packages/salt/renderers/stateconf.py", line 234, in render
    data = process_sls_data(sls_templ, tmplctx)
  File "/home/<user redacted>/.salt-ssh/lib/python3.6/site-packages/salt/renderers/stateconf.py", line 118, in process_sls_data
    **kws
TypeError: LoadedFunc object got multiple values for keyword argument 'context'

Setup
Install Salt v3004.1 locally in isolated virtualenv using pip install .. Remote host is accessible via SSH using public key auth, attempting to apply a stateconf state to the remote host using SSH, logging in with the same credential.

Local host is VMware VM running Ubuntu 18 using Python 3.6.9.

Target host is VMware VM in the same data center running Ubuntu 20 using Python 3.8.10

Steps to Reproduce the behavior
Run salt-ssh <target> state.sls fuz, where fuz/init.sls looks like this:

#!stateconf -o

.fuzfile:
  file.absent:
    - name: /tmp/myfuzfile

.sleeper:
  cmd.run:
    - name: sleep 180

Expected behavior
Remote host should apply the state without error. Works as expected using v2019.2 Minion.

Versions Report
Local host:

Salt Version:
          Salt: 3004.1
 
Dependency Versions:
          cffi: 1.14.6
      cherrypy: 18.6.1
      dateutil: 2.8.1
     docker-py: Not Installed
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 2.11.3
       libgit2: Not Installed
      M2Crypto: Not Installed
          Mako: Not Installed
       msgpack: 1.0.2
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     pycparser: 2.17
      pycrypto: Not Installed
  pycryptodome: 3.9.8
        pygit2: Not Installed
        Python: 3.6.9 (default, Nov  7 2019, 10:44:02)
  python-gnupg: 0.4.4
        PyYAML: 5.3.1
         PyZMQ: 18.0.1
         smmap: Not Installed
       timelib: 0.2.4
       Tornado: 4.5.3
           ZMQ: 4.3.1
 
System Versions:
          dist: ubuntu 18.04 bionic
        locale: UTF-8
       machine: x86_64
       release: 4.15.0-65-generic
        system: Linux
       version: Ubuntu 18.04 bionic

Remote host (using thindir: /usr/bin/python3.8 /var/tmp/.f7fd55/salt-call --versions-report)

Salt Version:
          Salt: 3004.1
 
Dependency Versions:
          cffi: Not Installed
      cherrypy: Not Installed
      dateutil: Not Installed
     docker-py: Not Installed
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 2.11.3
       libgit2: Not Installed
      M2Crypto: Not Installed
          Mako: Not Installed
       msgpack: 1.0.2
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     pycparser: Not Installed
      pycrypto: Not Installed
  pycryptodome: Not Installed
        pygit2: Not Installed
        Python: 3.8.10 (default, Mar 15 2022, 12:22:08)
  python-gnupg: Not Installed
        PyYAML: 5.3.1
         PyZMQ: Not Installed
         smmap: Not Installed
       timelib: Not Installed
       Tornado: 4.5.3
           ZMQ: Not Installed
 
System Versions:
          dist: ubuntu 20.04 focal
        locale: utf-8
       machine: x86_64
       release: 5.4.0-107-generic
        system: Linux
       version: Ubuntu 20.04 focal

Additional context
My knowledge of the inner workings of Salt is unfortunately somewhat limited, but this trivial patch appears to fix the issue:

$ git diff salt/renderers/stateconf.py
diff --git a/salt/renderers/stateconf.py b/salt/renderers/stateconf.py
index 3b0348633a..45943dc325 100644
--- a/salt/renderers/stateconf.py
+++ b/salt/renderers/stateconf.py
@@ -109,6 +109,10 @@ def render(input, saltenv="base", sls="", argline="", **kws):
         if context:
             ctx.update(context)
 
+        if 'context' in kws:
+            ctx.update(kws['context'])
+            del kws['context']
+
         tmplout = render_template(
             io.StringIO(data),
             saltenv,

However, this feels maybe a bit too good to be true.

Source tree is also using https://github.com/saltstack/salt/pull/61895 to fix a rendering bug with the import keyword.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in salt/renderers/stateconf.py, especially process_sls_data and render, and reproduce the failure with salt-ssh state.sls fuz using the minimal state shown. Done means the stateconf state renders and applies remotely on Salt 3004.1 without the duplicate-context TypeError.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
devops, infrastructure
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.