saltstack / saltstack/salt

onedir python packages removed from CentOS Stream 8 syndics after upgrading to 3005.1-4

Open
#63,675 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

needs-triage
Dominant language
Python
Stars
15.7k
Forks
5.6k
Avg merge
2d 44m
Merged PRs (30d)
80

Description

Description of Issue

We recently had a downtime on our compute cluster, where we use syndics to propagate Salt to compute nodes. During said downtime we were able to upgrade Salt from (onedir) 3005.1-3 to (onedir) 3005.1-4 on our syndics. We have four syndics in total, two running CentOS 7 and two running CentOS Stream 8 (the latter of which we intend to migrate to completely soon). We use GitPython as our gitfs and git_pillar provider.

After upgrading the Salt packages, the salt-minion and salt-syndic services came back cleanly on all four syndics. However the salt-master service on the CentOS Stream 8 syndics failed to start, with an error asking if GitPython was installed. After some digging; the site-packages/ directory (where Salt's pip places packages) is completely empty, whereas the correct packages were still installed on our CentOS 7 hosts where the salt-master service came back fine.

Using "salt-call pip.install GitPython" worked cleanly, after which the salt-master service was able to start successfully. Our concern is that the package update appeared to wipe away packages previously installed via pip, and that this may happen again with the next update.

Setup

Pulled from config file in /etc/salt/master.d/:
gitfs_provider: GitPython
git_pillar_provider: GitPython

Steps to Reproduce Issue

We have not tested, but in theory setting up a CentOS Stream 8 host on 3005.1-3 with GitPython etc. installed, then upgrading to 3005.1-4 should produce the same behavior.

Versions Report

From a CentOS Stream 8 syndic, now with GitPython (and pygit2) reinstalled:
Salt Version:
Salt: 3005.1

Dependency Versions:
cffi: 1.14.6
cherrypy: unknown
dateutil: 2.8.1
docker-py: Not Installed
gitdb: 4.0.10
gitpython: 3.1.30
Jinja2: 3.1.0
libgit2: 1.5.0
M2Crypto: Not Installed
Mako: Not Installed
msgpack: 1.0.2
msgpack-pure: Not Installed
mysql-python: Not Installed
pycparser: 2.21
pycrypto: Not Installed
pycryptodome: 3.9.8
pygit2: 1.11.1
Python: 3.9.16 (main, Jan 6 2023, 22:52:19)
python-gnupg: 0.4.8
PyYAML: 5.4.1
PyZMQ: 23.2.0
smmap: 5.0.0
timelib: 0.2.4
Tornado: 4.5.3
ZMQ: 4.3.4

System Versions:
dist: centos 8 n/a
locale: utf-8
machine: x86_64
release: 4.18.0-394.el8.x86_64
system: Linux
version: CentOS Stream 8 n/a

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by comparing the Salt 3005.1-3 and 3005.1-4 package update behavior on CentOS Stream 8, focusing on the Salt pip-installed site-packages directory and the salt-master service. Reproduce the upgrade with GitPython and pygit2 installed and check whether the packages disappear. Done means the cause and expected package-preservation behavior are established, with a verified fix or clearly documented limitation.

Written by the indexing model from the issue text.

Assessment

Tech stack
centos, git, python
Domain
devops, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.