saltstack / saltstack/salt

[BUG] 3005.1 onedir: pxgit2 ext_pillar configuration fails with ssl certificate rejection

Open
#62,869 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug needs-triage
Dominant language
Python
Stars
15.7k
Forks
5.6k
Avg merge
2d 44m
Merged PRs (30d)
80

Description

Description
Salt-Master is not connecting to a Gitlab instance with a letsencrypt certificate on a fresh onedir installation as an ext_pillar.

The same configuration is working with another salt-master, same version. The only difference being that the working one is a classic installation.

Noteworthy as well: The gitfs connection using the same Gitlab instance works.

If i uninstall the salt-master onedir version and install the classic version instead it works. No other changes are made at that point.

Setup

Please be as specific as possible and give set-up details.

  • on-prem machine
  • VM (KVM on a proxmox server)
  • VM running on a cloud service, please be explicit and add details
  • container (Kubernetes, Docker, containerd, etc. please specify)
  • or a combination, please be explicit
  • jails if it is FreeBSD
  • classic packaging
  • onedir packaging
  • used bootstrap to install

The git-Server in question is a Gitlab installation with a letsencrypt certificate.

For reference the used gitfs configuration works:

gitfs_remotes: 
  - https://gitlab.example.com/salt/states.git:
    - user: project_XXXX_bot
    - password: <authentication token>

But the corresponding ext_pillar configuration fails:

ext_pillar:
  - git:
    - __env__ https://gitlab.example.com/salt/pillars.git:
      - user: project_XXXX_bot
      - password: <authentication token>

I did not change the providers away from pygit2!

Steps to Reproduce the behavior
Produced Error :

[DEBUG   ] Set update lock for git_pillar remote '__env__ https://gitlab.example.com/salt/pillars.git'
[DEBUG   ] Fetching git_pillar remote '__env__ https://gitlab.example.com/salt/pillars.git'
[ERROR   ] Error occurred fetching git_pillar remote '__env__ https://gitlab.example.com/salt/pillars.git: user rejected certificate for gitlab.glaeserundflaschen.de
Traceback (most recent call last):
  File "salt/utils/gitfs.py", line 1870, in _fetch
    fetch_results = origin.fetch(**fetch_kwargs)
  File "/opt/saltstack/salt/pypath/lib/python3.9/site-packages/pygit2/remote.py", line 146, in fetch
    payload.check_error(err)
  File "/opt/saltstack/salt/pypath/lib/python3.9/site-packages/pygit2/callbacks.py", line 93, in check_error
    check_error(error_code)
  File "/opt/saltstack/salt/pypath/lib/python3.9/site-packages/pygit2/errors.py", line 65, in check_error
    raise GitError(message)
_pygit2.GitError: user rejected certificate for gluflab.glaeserundflaschen.de
[DEBUG   ] Removed update lock for git_pillar remote '__env__ https://gitlab.example.com/salt/pillars.git'
[DEBUG   ] This salt-master instance has accepted 1 minion keys.

Certificate on the gitlab instance is a valid letsencrypt wildcard certificate.

The only thing i did to get this error is installing the salt-master as a onedir installation.

Expected behavior
Behaving the same way as a classic installation.

Versions Report

salt --versions-report (Provided by running salt --versions-report. Please also mention any differences in master/minion versions.)
Salt Version:
          Salt: 3005.1
 
Dependency Versions:
          cffi: 1.14.6
      cherrypy: 18.6.1
      dateutil: 2.8.1
     docker-py: Not Installed
         gitdb: 4.0.9
     gitpython: 3.1.29
        Jinja2: 3.1.0
       libgit2: 1.5.0
      M2Crypto: Not Installed
          Mako: Not Installed
       msgpack: 1.0.2
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     pycparser: 2.21
      pycrypto: Not Installed
  pycryptodome: 3.9.8
        pygit2: 1.10.1
        Python: 3.9.14 (main, Oct  3 2022, 21:19:30)
  python-gnupg: 0.4.8
        PyYAML: 5.4.1
         PyZMQ: 23.2.0
         smmap: 5.0.0
       timelib: 0.2.4
       Tornado: 4.5.3
           ZMQ: 4.3.4
 
System Versions:
          dist: debian 11 bullseye
        locale: utf-8
       machine: x86_64
       release: 5.10.0-18-amd64
        system: Linux
       version: Debian GNU/Linux 11 bullseye

For comparison: The working master:

Salt Version:
          Salt: 3005.1
 
Dependency Versions:
          cffi: Not Installed
      cherrypy: unknown
      dateutil: 2.8.1
     docker-py: Not Installed
         gitdb: 4.0.5
     gitpython: 3.1.14
        Jinja2: 2.11.3
       libgit2: 1.1.0
      M2Crypto: Not Installed
          Mako: Not Installed
       msgpack: 1.0.0
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     pycparser: Not Installed
      pycrypto: Not Installed
  pycryptodome: 3.9.7
        pygit2: 1.4.0
        Python: 3.9.2 (default, Feb 28 2021, 17:03:44)
  python-gnupg: Not Installed
        PyYAML: 5.3.1
         PyZMQ: 20.0.0
         smmap: 4.0.0
       timelib: Not Installed
       Tornado: 4.5.3
           ZMQ: 4.3.4
 
System Versions:
          dist: debian 11 bullseye
        locale: utf-8
       machine: x86_64
       release: 5.10.0-18-amd64
        system: Linux
       version: Debian GNU/Linux 11 bullseye
 

Additional context

Comparing the versions on the two masters i changed the pygit2 version in the onedir installation:

salt-pip uninstall pygit2
salt-pip install pygit2==1.4.0

After this the master startet working as expected. So it seems that something significantly changed from pygit2 1.4.0 to 1.10.1.

I'm currently using this as a workaround.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in salt/utils/gitfs.py around _fetch, then inspect the git_pillar/ext_pillar path and the reported pygit2 1.10.1 versus 1.4.0 behavior. Reproduce the onedir configuration with the LetsEncrypt GitLab certificate and verify that ext_pillar handles it like gitfs and the classic installation.

Written by the indexing model from the issue text.

Assessment

Tech stack
git, python
Domain
devops, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.