salt-minion starts up many subporcess
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 15.7k
- Forks
- 5.6k
- Avg merge
- 2d 44m
- Merged PRs (30d)
- 80
Description
Description of Issue
I recieved a CPU overload alert, and find out that my minion server is running hundreds of salt-minion. They were started up by main process of salt-minion service at different time. And the subprocess still exist even i kill the main process.
I have no idea why it happeds. In other minion server, only one salt-minion process is found. How can I avoid it happends again ?
Setup
/etc/salt/minonfile has only one config, that ismaster:- and this is systemd service unit configuration
[Unit] Description=The Salt Minion Documentation=man:salt-minion(1) file:///usr/share/doc/salt/html/contents.html https://docs.saltstack.com/en/latest/contents.html After=network.target salt-master.service [Service] KillMode=process Type=notify NotifyAccess=all LimitNOFILE=8192 ExecStart=/usr/bin/salt-minion [Install] WantedBy=multi-user.target
Steps to Reproduce Issue
- all PPIDs of subprocess is 1, and main process 5475 showed by
systemctl status salt-minionis running too.


- afiter running
kill -9 5475, the service became failed,but subprocess were still existing by usingps -ef

Versions Report
master is 2018.3.3, and minion is 2018.3.4
Salt Version:
Salt: 2018.3.3
Dependency Versions:
cffi: 1.12.3
cherrypy: unknown
dateutil: Not Installed
docker-py: Not Installed
gitdb: Not Installed
gitpython: Not Installed
ioflo: Not Installed
Jinja2: 2.9.6
libgit2: Not Installed
libnacl: Not Installed
M2Crypto: 0.21.1
Mako: Not Installed
msgpack-pure: Not Installed
msgpack-python: 0.5.6
mysql-python: Not Installed
pycparser: 2.19
pycrypto: 2.6.1
pycryptodome: Not Installed
pygit2: Not Installed
Python: 2.7.5 (default, Oct 30 2018, 23:45:53)
python-gnupg: Not Installed
PyYAML: 3.11
PyZMQ: 15.3.0
RAET: Not Installed
smmap: Not Installed
timelib: Not Installed
Tornado: 4.2.1
ZMQ: 4.1.4
System Versions:
dist: centos 7.4.1708 Core
locale: UTF-8
machine: x86_64
release: 3.10.0-693.2.2.el7.x86_64
system: Linux
version: CentOS Linux 7.4.1708 Core
And salt-minion is version is 2018.3.4 (Oxygen)
Salt Version:
Salt: 2018.3.4
Dependency Versions:
cffi: Not Installed
cherrypy: Not Installed
dateutil: Not Installed
docker-py: Not Installed
gitdb: Not Installed
gitpython: Not Installed
ioflo: Not Installed
Jinja2: 2.10
libgit2: Not Installed
libnacl: Not Installed
M2Crypto: Not Installed
Mako: Not Installed
msgpack-pure: Not Installed
msgpack-python: 0.5.6
mysql-python: Not Installed
pycparser: Not Installed
pycrypto: 2.6.1
pycryptodome: Not Installed
pygit2: Not Installed
Python: 2.7.5 (default, Oct 30 2018, 23:45:53)
python-gnupg: Not Installed
PyYAML: 3.11
PyZMQ: 15.3.0
RAET: Not Installed
smmap: Not Installed
timelib: Not Installed
Tornado: 4.2.1
ZMQ: 4.1.4
System Versions:
dist: centos 7.6.1810 Core
locale: UTF-8
machine: x86_64
release: 3.10.0-957.5.1.el7.x86_64
system: Linux
version: CentOS Linux 7.6.1810 Core
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the salt-minion systemd unit, especially KillMode=process, alongside the reported /etc/salt/minon configuration and the salt-minion service entry point. Reproduce the process-tree behavior with systemctl status salt-minion and ps -ef, then determine why subprocesses remain after the main process exits. Done means the cause is identified and the service no longer leaves unintended subprocesses behind.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- devops, infrastructure, operating-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100