saltstack / saltstack/salt

[BUG] CPU usage (FreeBSD kevent/kqueue)

Open
#61,333 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Question
Dominant language
Python
Stars
15.7k
Forks
5.6k
Avg merge
2d 44m
Merged PRs (30d)
80

Description

I have a FreeBSD host with many jails. In each jail a salt-minion is installed. Although each minion consumes something like 0.1% CPU it starts to have an impact on the load average.

Running truss -p <pid_of_minion> I noticed the following:

kevent(9,0x0,0,{ },1000,{ 0.640124878 })         = 0 (0x0)
getpid()                                         = 18245 (0x4745)
kevent(9,0x0,0,{ },1000,{ 0.958296299 })         = 0 (0x0)
getpid()                                         = 18245 (0x4745)
kevent(9,0x0,0,{ },1000,{ 0.963230610 })         = 0 (0x0)
getpid()                                         = 18245 (0x4745)
kevent(9,0x0,0,{ },1000,{ 0.973388911 })         = 0 (0x0)
getpid()                                         = 18245 (0x4745)
(...)

It happens every second (kevent followed by a getpid).
I'm wondering if this could be the cause of CPU usage and why should it be run every second? Is it configurable?

This is on FreeBSD 13.0-RELEASE with:

chimay% salt --versions-report
Salt Version:
          Salt: 3004
 
Dependency Versions:
          cffi: 1.14.6
      cherrypy: Not Installed
      dateutil: Not Installed
     docker-py: Not Installed
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 3.0.1
       libgit2: Not Installed
      M2Crypto: Not Installed
          Mako: Not Installed
       msgpack: 1.0.2
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     pycparser: 2.20
      pycrypto: Not Installed
  pycryptodome: 3.10.1
        pygit2: Not Installed
        Python: 3.8.12 (default, Nov 27 2021, 21:04:30)
  python-gnupg: Not Installed
        PyYAML: 5.4.1
         PyZMQ: 22.3.0
         smmap: Not Installed
       timelib: Not Installed
       Tornado: 4.5.3
           ZMQ: 4.3.4
 
System Versions:
          dist: freebsd 13.0 
        locale: utf-8
       machine: amd64
       release: 13.0-RELEASE-p4
        system: FreeBSD
       version: FreeBSD 13.0 



Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the behavior on FreeBSD 13.0 with a Salt 3004 minion and inspect the kevent/getpid sequence shown by truss. Trace the minion's event-loop path to determine why it wakes every second and whether an existing configuration controls that interval. Done means identifying the cause and documenting or testing a concrete correction.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
operating-systems, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.