saltstack / saltstack/salt

[BUG] salt-minion 3004 upgrade from salt-master produces "The renderer jinja|yaml is unavailable"

Open
#61,462 12 comments 11 reactions 1 assignee View on GitHub

@cmcmarrow is already working on this.

Since Jan 15, 2022.

bug Confirmed
Dominant language
Python
Stars
15.7k
Forks
5.6k
Avg merge
2d 44m
Merged PRs (30d)
80

Description

Description
I have been attempting to upgrade Debian 10/11 machines to salt-minion/salt-common 3004 recently. I would like to be able to perform this upgrade through the salt-master if possible. However, whenever I kick-off an upgrade of the salt-minion through salt itself, I receive the error: salt.exceptions.LoaderError: The renderer jinja|yaml is unavailable, this error is often because the needed software is unavailable

I'm facing a similar problem to https://github.com/saltstack/salt/issues/49834 ; however, these machines are all using systemd.

Setup
salt-master is an LXC container running debian 10 (buster), provisioned directly from the proxmox debian 10 template
salt-minion is also and LXC container running debian 10, provisioned in the same way.

I have taken the advice of the FAQ in developing a state that waits until the upgrade is complete and restarts the salt-minion service at a time when it will be less disruptive, rather than letting apt handle the restart of the service:

{%- if grains['os_family'] == 'Debian' %}

Disable starting services:
  file.managed:
    - name: /usr/sbin/policy-rc.d
    - user: root
    - group: root
    - mode: 0755
    - contents:
      - '#!/bin/sh'
      - exit 101
    # do not touch if already exists
    - replace: False
    - prereq:
      - pkg: Upgrade Salt Minion

{%- endif %}

Upgrade Salt Minion:
  pkg.latest:
    - name: salt-minion
    - refresh: True

Enable Salt Minion:
  service.enabled:
    - name: salt-minion
    - require:
      - pkg: Upgrade Salt Minion

{%- if grains['os_family'] == 'Debian' %}

Enable starting services:
  file.absent:
    - name: /usr/sbin/policy-rc.d
    - onchanges:
      - pkg: Upgrade Salt Minion

{%- endif %}

{%- if grains['os'] != 'Windows' %}
Restart Salt Minion:
  cmd.run:
    - name: 'salt-call service.restart salt-minion'
    - bg: True
    - onchanges:
      - pkg: Upgrade Salt Minion
{%- endif %}

I then call this state from the salt-master, while specifying the node to update. I have edited the apt source.list by hand to reference Buster 10 with 3004. The machine is currently running 3003, but I have reproduced the bug by upgrading from 3002 as well:

salt node-1 state.apply maintenance.update-salt

Steps to Reproduce the behavior

  • Install salt 3002 or 3003 on debian buster
  • Update source list to prepare for the upgrade to 3004
  • Use the above state file to upgrade the target node from salt-master
  • Fail
    The minion function caused an exception: Traceback (most recent call last):
      File "/usr/lib/python3/dist-packages/salt/minion.py", line 1905, in _thread_return
        for executor in executors
      File "/usr/lib/python3/dist-packages/salt/minion.py", line 1861, in _execute_job_function
        executors[-1] = "sudo"  # replace the last one with sudo
      File "/usr/lib/python3/dist-packages/salt/loader.py", line 1241, in __call__
      File "/usr/lib/python3/dist-packages/salt/loader.py", line 2274, in run
      File "/usr/lib/python3/dist-packages/salt/loader.py", line 2289, in _run_as
      File "/usr/lib/python3/dist-packages/salt/executors/direct_call.py", line 12, in execute
      File "/usr/lib/python3/dist-packages/salt/loader.py", line 1241, in __call__
      File "/usr/lib/python3/dist-packages/salt/loader.py", line 2274, in run
      File "/usr/lib/python3/dist-packages/salt/loader.py", line 2289, in _run_as
      File "/usr/lib/python3/dist-packages/salt/modules/state.py", line 792, in apply_
        if mods:
      File "/usr/lib/python3/dist-packages/salt/modules/state.py", line 1391, in sls
        snapper_pre = _snapper_pre(opts, kwargs.get("__pub_jid", "called localy"))
      File "/usr/lib/python3/dist-packages/salt/state.py", line 3269, in call_high
        # Verify that the high data is structurally sound
      File "/usr/lib/python3/dist-packages/salt/state.py", line 2495, in call_chunks
        if action == "kill":
      File "/usr/lib/python3/dist-packages/salt/state.py", line 2999, in call_chunk
        elif status == "met":
      File "/usr/lib/python3/dist-packages/salt/utils/decorators/state.py", line 45, in _func
        result = func(*args, **kwargs)
      File "/usr/lib/python3/dist-packages/salt/state.py", line 2229, in call
        if low.get("__prereq__"):
      File "/usr/lib/python3/dist-packages/salt/state.py", line 1256, in check_refresh
        return
      File "/usr/lib/python3/dist-packages/salt/state.py", line 1213, in module_refresh
        Refresh all the modules
      File "/usr/lib/python3/dist-packages/salt/state.py", line 1191, in load_modules
        providers = {}
      File "/usr/lib/python3/dist-packages/salt/loader.py", line 738, in render
    salt.exceptions.LoaderError: The renderer jinja|yaml is unavailable, this error is often because the needed software is unavailable

The minion is not restarted. After this error, the master is unable to communicate with under any circumstance until the minion is manually restarted with systemctl restart salt-minion directly on the node.

Expected behavior
I expect that the salt-minion upgrade doesn't fail and receives the restart signal cleanly

Screenshots
n/a

Versions Report

salt --versions-report Master -- minion is running 3003, attempting to upgrade to 3004 ``` Salt Version: Salt: 3004

Dependency Versions:
cffi: Not Installed
cherrypy: Not Installed
dateutil: 2.8.1
docker-py: Not Installed
gitdb: 4.0.5
gitpython: 3.1.14
Jinja2: 2.11.3
libgit2: Not Installed
M2Crypto: Not Installed
Mako: Not Installed
msgpack: 1.0.0
msgpack-pure: Not Installed
mysql-python: Not Installed
pycparser: Not Installed
pycrypto: Not Installed
pycryptodome: 3.9.7
pygit2: Not Installed
Python: 3.9.2 (default, Feb 28 2021, 17:03:44)
python-gnupg: 0.4.6
PyYAML: 5.3.1
PyZMQ: 20.0.0
smmap: 4.0.0
timelib: Not Installed
Tornado: 4.5.3
ZMQ: 4.3.4

System Versions:
dist: debian 11 bullseye
locale: utf-8
machine: x86_64
release: 5.13.19-2-pve
system: Linux
version: Debian GNU/Linux 11 bullseye

</details>

**Additional context**

- No errors at all when upgrading directly on the minion. Salt upgrades fine and can communicate with no issues directly to the master once finished. However, this is less than ideal if you plan to upgrade many minions to 3004 and would like to do it centrally.

- Upgrading from 3002 -> 3003 does not produce this error (even when the master is running 3004). This bug appears to be specific to the 3004 upgrade in my case. 3002 -> 3004 produces the error.

- I can reproduce the error using Bullseye going from deb repo (3002.6) -> salt repo (3004), so it does not appear to be specific to Buster

- I'm dubious that LXC is affecting this error, since 3002 -> 3003 shows that success is possible

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.