saltstack / saltstack/salt

RHEL8 minion does not run states after installation of minion

Open
#63,223 10 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug needs-triage Salt-Cloud Transport
Dominant language
Python
Stars
15.7k
Forks
5.6k
Avg merge
2d 44m
Merged PRs (30d)
80

Description

Description
Creating a new RHEL8 salt-minion using salt-cloud. This automatically installs the latest salt-minion version for 3005.x

Post the salt configuration, the minion does not execute any state passed from the master. The error in the minion log says

2022-12-06 16:21:18,435 [salt.utils.event :821 ][DEBUG ][4602] Sending event: tag = _salt_error; data = {'message': 'The minion function caused an exception', 'args': ('The minion function caused an exception',)

I am attaching a debug based output of the minion log to this report so that all the details are available.

Setup

Fresh install of salt-master 3005.1-2 on RHEL9. Trying to use salt-cloud to create a new RHEL8 minion (RHEL 8.0)

The platform is VMware and the salt-master and minion are both VMware virtual machines.

There is no firewall running on either master or minion.

Both are in the same network, so no VLAN etc configuration

Trying to use onedir 3005 as bootstrap arguments. Same thing happens with any method for the minion install git/stable etc

Please be as specific as possible and give set-up details.

  • on-prem machine
  • [ X] VM (VMware 7.x/Vsphere 7)
  • [ X] classic packaging
  • [ X] onedir packaging
  • used bootstrap to install - this is running via salt-cloud command arguments.

Steps to Reproduce the behavior
Attaching the logs to this report.

Steps:

  • Fresh install of salt-master with salt-cloud version 3005.1-2 on RHEL9
  • create a configuration for a new RHEL8 system and call the salt-cloud command to create the VM
  • After the minion spins up and salt installs on it, no state will work on the minion. The above errors mentioned are seen.
  • Basic module commands from master to ping the minion etc will work. However any state execution via sls will hang indefinitely and return with a timeout. The errors will get thrown in the minion logs.

Expected behavior

The state should execute successfully. Never happens

Screenshots
If applicable, add screenshots to help explain your problem.

Versions Report

$ salt-master --versions-report Salt Version: Salt: 3005.1

Dependency Versions:
cffi: 1.14.5
cherrypy: Not Installed
dateutil: 2.8.1
docker-py: Not Installed
gitdb: Not Installed
gitpython: Not Installed
Jinja2: 2.11.3
libgit2: 1.3.0
M2Crypto: 0.38.0
Mako: Not Installed
msgpack: 1.0.3
msgpack-pure: Not Installed
mysql-python: Not Installed
pycparser: 2.20
pycrypto: 3.16.0
pycryptodome: 3.14.0
pygit2: 1.7.1
Python: 3.9.14 (main, Nov 7 2022, 00:00:00)
python-gnupg: Not Installed
PyYAML: 5.4.1
PyZMQ: 22.3.0
smmap: Not Installed
timelib: Not Installed
Tornado: 4.5.3
ZMQ: 4.3.4

System Versions:
dist: rhel 9.0 Plow
locale: utf-8
machine: x86_64
release: 5.14.0-70.13.1.el9_0.x86_64
system: Linux
version: Red Hat Enterprise Linux 9.0 Plow

salt-minion --versions-report

Salt Version:
Salt: 3005.1

Dependency Versions:
cffi: 1.14.6
cherrypy: 18.6.1
dateutil: 2.8.1
docker-py: Not Installed
gitdb: Not Installed
gitpython: Not Installed
Jinja2: 3.1.0
libgit2: Not Installed
M2Crypto: Not Installed
Mako: Not Installed
msgpack: 1.0.2
msgpack-pure: Not Installed
mysql-python: Not Installed
pycparser: 2.21
pycrypto: Not Installed
pycryptodome: 3.9.8
pygit2: Not Installed
Python: 3.9.15 (main, Nov 8 2022, 03:47:03)
python-gnupg: 0.4.8
PyYAML: 5.4.1
PyZMQ: 23.2.0
smmap: Not Installed
timelib: 0.2.4
Tornado: 4.5.3
ZMQ: 4.3.4

System Versions:
dist: rhel 8.0 Ootpa
locale: utf-8
machine: x86_64
release: 4.18.0-80.el8.x86_64
system: Linux
version: Red Hat Enterprise Linux 8.0 Ootpa

(Provided by running salt --versions-report. Please also mention any differences in master/minion versions.) Both master and minion are at same level.
any simple sls file can be tried.  Say a small sls to install gcc.  In our case, we have a startup state to rename the host based on the VM name in Vmware.
[salt-master-exception.txt](https://github.com/saltstack/salt/files/10169518/salt-master-exception.txt)
[salt-minion-log.txt](https://github.com/saltstack/salt/files/10169521/salt-minion-log.txt)

Additional context
We have a master running 3003 version of salt and everything works fine there. This seems to be 3005 specific. We need 3005 to move our master to the newer version. But this is breaking.

Please let me know if you need any other information from me.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the attached salt-master-exception.txt and salt-minion-log.txt alongside the RHEL8/RHEL9 and Salt 3005.1 setup described in the report. Trace why state execution hangs while basic module commands succeed; done means a freshly installed RHEL8 minion executes a simple SLS state successfully without timing out.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
devops, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.