saltstack / saltstack/salt

[BUG] Critical bug in salt [3001] whith git remote backend and `gitpython`

Open
#57,729 6 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

bug help-wanted P2 Regression severity-medium
Dominant language
Python
Stars
15.7k
Forks
5.6k
Avg merge
2d 44m
Merged PRs (30d)
80

Description

Description

Following this bug, I have noticed many other bugs while refreshing or synchronizing stuffs in SaltStack 3000+ with a git backend.

Indeed, I have custom modules. These modules were not synchronized this morning. I had to force the synchronization with salt '*' saltutil.sync_modules.

However, then, I had a really serious bug with my pillars. I was not able to synchronize these informations anymore. I was wondering if this was related to the way I am synhronizing those pillars with git backend using GitPython, so I switched to pygit2.

The problem on this master is gone but is still present on masters using GitPython.

Setup
Create a pillar sls file and try to synchronize it through gitfs backend and gitpython with, at least, salt-master 3000.3 and 3001. I am not sure, but it seems that old 2019.X are not affected.

Versions Report

salt --versions-report Salt Version: Salt: 3000.3

Dependency Versions:
cffi: Not Installed
cherrypy: Not Installed
dateutil: 2.6.1
docker-py: Not Installed
gitdb: 2.0.3
gitpython: 2.1.8
Jinja2: 2.10
libgit2: Not Installed
M2Crypto: Not Installed
Mako: Not Installed
msgpack-pure: Not Installed
msgpack-python: 0.5.6
mysql-python: Not Installed
pycparser: Not Installed
pycrypto: 2.6.1
pycryptodome: Not Installed
pygit2: Not Installed
Python: 3.6.9 (default, Apr 18 2020, 01:56:04)
python-gnupg: 0.4.1
PyYAML: 3.12
PyZMQ: 16.0.2
smmap: 2.0.3
timelib: Not Installed
Tornado: 4.5.3
ZMQ: 4.2.5

System Versions:
dist: Ubuntu 18.04 bionic
locale: ANSI_X3.4-1968
machine: x86_64
release: 5.4.41-1-pve
system: Linux
version: Ubuntu 18.04 bionic

As already said, this bug also affects salt-master 3001.

Upgrading configuration and software in order to use pygit2 solves this issue:

salt --versions-report Salt Version: Salt: 3001

Dependency Versions:
cffi: 1.14.0
cherrypy: Not Installed
dateutil: 2.6.1
docker-py: Not Installed
gitdb: 2.0.3
gitpython: 2.1.8
Jinja2: 2.10
libgit2: 1.0.0
M2Crypto: Not Installed
Mako: Not Installed
msgpack-pure: Not Installed
msgpack-python: 0.5.6
mysql-python: Not Installed
pycparser: 2.20
pycrypto: 2.6.1
pycryptodome: 3.4.7
pygit2: 1.2.1
Python: 3.6.9 (default, Apr 18 2020, 01:56:04)
python-gnupg: 0.4.1
PyYAML: 3.12
PyZMQ: 17.1.2
smmap: 2.0.3
timelib: Not Installed
Tornado: 4.5.3
ZMQ: 4.2.5

System Versions:
dist: ubuntu 18.04 Bionic Beaver
locale: UTF-8
machine: x86_64
release: 4.15.0-50-generic
system: Linux
version: Ubuntu 18.04 Bionic Beaver

Best regards,
Rémy

edit : it only affects salt 3001 and not 3000.3

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing gitfs synchronization with GitPython on Salt 3001, including saltutil.sync_modules and pillar synchronization, using the reported Ubuntu and dependency versions. Compare the behavior with pygit2 and the stated Salt 3000.3 setup. Done means the affected custom modules and pillars synchronize successfully through the GitPython backend.

Written by the indexing model from the issue text.

Assessment

Tech stack
git, python
Domain
backend, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.