saltstack / saltstack/salt

[BUG] LazyLoader._refresh_file_mapping is called for every load miss

Open
#59,265 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug severity-medium
Dominant language
Python
Stars
15.7k
Forks
5.6k
Avg merge
2d 44m
Merged PRs (30d)
80

Description

Description

I'm working on tracing for Salt and I noticed a peculiar thing:

image

Here's the part of highstate that produced the trace above:

less:
  pkg.installed:
    - version: latest
    - require:
      - sls: apt

htop:
  pkg.installed:
    - version: latest
    - require:
      - sls: apt

You may notice that pkg.check_db is loaded for both states and it takes 25ms. The same applies to other pkg related functions. The thing is: pkg.check_db is only implemented by salt.modules.ebuildpkg when running on Gentoo, but I'm using salt.modules.aptpkg, since I'm running Debian.

Let's expand the trace for less:

image

Now it turns out that most of the time is spent in salt.loader.LazyLoader._refresh_file_mapping, which is getting called every time a non-implemented function is requested.

This time really adds up:

  • 190ms total to enforce pkg.installed for less
  • 65ms to run apt-cache and make sure it's already on the latest (actual work)
  • 112ms to try load unloadable functions and refresh file mappings 4 times

That's 60% of time spent on fruitless activities.

Expected behavior

Salt doesn't call salt.loader.LazyLoader._refresh_file_mapping repeatedly when there is no reason to do so.

The simplest way for pkg module is not to call Gentoo specific functions and put these calls into salt.modules.ebuildpkg directly. The proper way is to implement negative caching of some sort.

Versions Report

salt --versions-report
Salt Version:
          Salt: 3002.2

Dependency Versions:
          cffi: Not Installed
      cherrypy: Not Installed
      dateutil: 2.8.1
     docker-py: 4.1.0
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 2.11.2
       libgit2: Not Installed
      M2Crypto: Not Installed
          Mako: Not Installed
       msgpack: 1.0.0
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     pycparser: Not Installed
      pycrypto: Not Installed
  pycryptodome: 3.9.7
        pygit2: Not Installed
        Python: 3.9.1 (default, Dec  8 2020, 07:51:42)
  python-gnupg: Not Installed
        PyYAML: 5.3.1
         PyZMQ: 20.0.0
         smmap: Not Installed
       timelib: Not Installed
       Tornado: 4.5.3
           ZMQ: 4.3.3

System Versions:
          dist: debian testing bullseye
        locale: utf-8
       machine: aarch64
       release: 5.11.0-rc2-ivan
        system: Linux
       version: Debian GNU/Linux testing bullseye

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in salt.loader.LazyLoader._refresh_file_mapping and trace the load-miss path described in the issue, then compare the pkg calls made through salt.modules.aptpkg and salt.modules.ebuildpkg. Reproduce the repeated refreshes with the provided pkg.installed highstate example and verify that missing functions no longer trigger unnecessary file-mapping refreshes.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
devops, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.