[Bug]: pip.installed slow for already installed packages
Open
Nobody has claimed this yet.
bug
needs-triage
- Dominant language
- Python
- Stars
- 15.7k
- Forks
- 5.6k
- Avg merge
- 2d 44m
- Merged PRs (30d)
- 80
Description
What happened?
Description
Copy of issue #66946 which still exists.
The pip.installed state is slow when run for already installed package. It takes 400-700ms per state on various hosts I have tried. This is a problem when re-running a salt top file to apply a single change when there are a lot of pip packages that are managed. Re-running the state.apply should take a few seconds yet can take much longer depending on the number of python packages.
Setup
Please be as specific as possible and give set-up details.
- VM running on a cloud service (AWS ec2 instance rhel 8.10)
- container (ubuntu 22 docker)
- onedir packaging
- masterless
Steps to Reproduce the behavior
-
Setup directories and files; set minion to masterless; use state file that installs already installed python package:
cat > /etc/salt/minion <<'EOF' file_client: local master_type: disable EOF mkdir /srv/salt cat > /srv/salt/top.sls <<'EOF' base: '*': - test EOF cat > /srv/salt/test.sls <<'EOF' {% for i in range(100) %} "pip package {{i}}": pip.installed: - name: requests {% endfor %} EOF -
Run state.apply which takes over a minute instead of a few seconds
time salt-call --local state.apply
# time salt-call --local state.apply
local:
----------
ID: pip package 0
Function: pip.installed
Name: requests
Result: True
Comment: Python package requests was already installed
All specified packages are already installed
Started: 02:15:32.235629
Duration: 1491.119 ms
Changes:
----------
ID: pip package 1
Function: pip.installed
Name: requests
Result: True
Comment: Python package requests was already installed
All specified packages are already installed
Started: 02:15:33.726885
Duration: 651.78 ms
Changes:
----------
ID: pip package 2
Function: pip.installed
Name: requests
Result: True
Comment: Python package requests was already installed
All specified packages are already installed
Started: 02:15:34.378789
Duration: 713.077 ms
Changes:
----------
ID: pip package 3
Function: pip.installed
Name: requests
Result: True
Comment: Python package requests was already installed
All specified packages are already installed
Started: 02:15:35.091983
Duration: 629.822 ms
Changes:
----------
ID: pip package 4
Function: pip.installed
Name: requests
Result: True
Comment: Python package requests was already installed
All specified packages are already installed
Started: 02:15:35.721973
Duration: 658.011 ms
Changes:
----------
ID: pip package 5
Function: pip.installed
Name: requests
Result: True
Comment: Python package requests was already installed
All specified packages are already installed
Started: 02:15:36.380101
Duration: 601.251 ms
Changes:
----------
ID: pip package 6
Function: pip.installed
Name: requests
Result: True
Comment: Python package requests was already installed
All specified packages are already installed
Started: 02:15:36.981481
Duration: 583.015 ms
Changes:
----------
ID: pip package 7
Function: pip.installed
Name: requests
Result: True
Comment: Python package requests was already installed
All specified packages are already installed
Started: 02:15:37.564626
Duration: 752.305 ms
Changes:
----------
ID: pip package 8
Function: pip.installed
Name: requests
Result: True
Comment: Python package requests was already installed
All specified packages are already installed
Started: 02:15:38.317096
Duration: 1048.174 ms
Changes:
----------
ID: pip package 9
Function: pip.installed
Name: requests
Result: True
Comment: Python package requests was already installed
All specified packages are already installed
Started: 02:15:39.365787
Duration: 949.108 ms
Changes:
----------
ID: pip package 10
Function: pip.installed
Name: requests
Result: True
Comment: Python package requests was already installed
All specified packages are already installed
Started: 02:15:40.315024
Duration: 682.177 ms
Changes:
----------
...
----------
ID: pip package 90
Function: pip.installed
Name: requests
Result: True
Comment: Python package requests was already installed
All specified packages are already installed
Started: 02:16:44.127580
Duration: 627.173 ms
Changes:
----------
ID: pip package 91
Function: pip.installed
Name: requests
Result: True
Comment: Python package requests was already installed
All specified packages are already installed
Started: 02:16:44.754875
Duration: 648.814 ms
Changes:
----------
ID: pip package 92
Function: pip.installed
Name: requests
Result: True
Comment: Python package requests was already installed
All specified packages are already installed
Started: 02:16:45.403811
Duration: 672.239 ms
Changes:
----------
ID: pip package 93
Function: pip.installed
Name: requests
Result: True
Comment: Python package requests was already installed
All specified packages are already installed
Started: 02:16:46.076216
Duration: 672.331 ms
Changes:
----------
ID: pip package 94
Function: pip.installed
Name: requests
Result: True
Comment: Python package requests was already installed
All specified packages are already installed
Started: 02:16:46.748688
Duration: 657.274 ms
Changes:
----------
ID: pip package 95
Function: pip.installed
Name: requests
Result: True
Comment: Python package requests was already installed
All specified packages are already installed
Started: 02:16:47.406082
Duration: 609.847 ms
Changes:
----------
ID: pip package 96
Function: pip.installed
Name: requests
Result: True
Comment: Python package requests was already installed
All specified packages are already installed
Started: 02:16:48.016126
Duration: 680.64 ms
Changes:
----------
ID: pip package 97
Function: pip.installed
Name: requests
Result: True
Comment: Python package requests was already installed
All specified packages are already installed
Started: 02:16:48.696929
Duration: 671.398 ms
Changes:
----------
ID: pip package 98
Function: pip.installed
Name: requests
Result: True
Comment: Python package requests was already installed
All specified packages are already installed
Started: 02:16:49.368526
Duration: 639.387 ms
Changes:
----------
ID: pip package 99
Function: pip.installed
Name: requests
Result: True
Comment: Python package requests was already installed
All specified packages are already installed
Started: 02:16:50.008033
Duration: 655.247 ms
Changes:
Summary for local
--------------
Succeeded: 100
Failed: 0
--------------
Total states run: 100
Total run time: 78.412 s
real 1m19.820s
user 1m4.434s
sys 0m12.546s
Expected behavior
The total time should only be a few seconds when the system is already configured.
Additional context
Adding caching to the list of installed python packages is one option for a possible solution that could help a lot.
Type of salt install
Official rpm
Major version
3007.x
What supported OS are you seeing the problem on? Can select multiple. (If bug appears on an unsupported OS, please open a GitHub Discussion instead)
rhel-8, rhel-9, rhel-10, rockylinux-8, ubuntu-22.04, rockylinux-9, rockylinux-10
salt --versions-report output
Salt Version:
Salt: 3008.2
Python Version:
Python: 3.14.6 (main, Jun 11 2026, 02:19:05) [GCC 11.2.0]
Dependency Versions:
cffi: 2.0.0
cherrypy: 18.10.0
cryptography: 48.0.0
dateutil: 2.9.0.post0
docker-py: Not Installed
gitdb: 4.0.12
gitpython: Not Installed
Jinja2: 3.1.6
libgit2: Not Installed
looseversion: 1.3.0
M2Crypto: Not Installed
Mako: Not Installed
msgpack: 1.1.2
msgpack-pure: Not Installed
mysql-python: Not Installed
packaging: 24.0
pycparser: 3.00
pycrypto: Not Installed
pycryptodome: 3.23.0
pygit2: Not Installed
python-gnupg: 0.5.6
PyYAML: 6.0.3
PyZMQ: 27.1.0
relenv: 0.22.14
smmap: 5.0.2
timelib: 0.3.0
Tornado: 6.5.7
ZMQ: 4.3.5
Salt Package Information:
Package Type: onedir
System Versions:
dist: rocky 10.0 Red Quartz
locale: utf-8
machine: x86_64
release: 6.12.0-55.32.1.el10_0.x86_64
system: Linux
version: Rocky Linux 10.0 Red Quartz
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the pip.installed state entry point and reproduce the issue with the provided masterless setup and salt-call --local state.apply command. Compare repeated already-installed package states with the reported 100-state run; done means the configured system completes in a few seconds without changing the successful result.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- devops, infrastructure, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100