saltstack / saltstack/salt

[Bug]: Intermittent ext_pillar failure: 'Pygit2' object has no attribute 'repo'

Open
#70,081 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug needs-triage
Dominant language
Python
Stars
15.7k
Forks
5.6k
Avg merge
2d 44m
Merged PRs (30d)
80

Description

What happened?
Description

We're seeing ext_pillar intermittently fail to render when using the pygit2 provider.
Every third or fourth highstate fails; the runs in between succeed with no config change.
The error is always (on salt-minion):

Data failed to compile:
----------
Pillar failed to render with the following messages:
----------
Failed to load ext_pillar git: 'Pygit2' object has no attribute 'repo'

It's not tied to a specific minion or a specific remote — the same setup
just works on one run and fails on the next.

Setup
  • Salt master (installed via official RPM), CentOS Stream 9
  • gitfs and ext_pillar with several remotes over HTTPS, each using its own per-remote user/password
  • gitfs_provider and ext_pillar_provider: pygit2
Steps to Reproduce
  1. Configure ext_pillar with one or more remotes using pygit2 as provider.
  2. Run highstate or state.apply repeatedly against a minion.
Expected behavior

highstate should succeed on every run.

Actual behavior

highstate run fails intermittently with
'Pygit2' object has no attribute 'repo'. Full traceback from the master log:

2026-08-18 09:33:45,925 [salt.pillar      :1139][ERROR   ][57987] Exception caught loading ext_pillar 'git':
File "/opt/saltstack/salt/lib/python3.14/site-packages/salt/pillar/__init__.py", line 1131, in ext_pillar
  ext = self._external_pillar_data(pillar, val, key)
File "/opt/saltstack/salt/lib/python3.14/site-packages/salt/pillar/__init__.py", line 1059, in _external_pillar_data
  ext = self.ext_pillars[key](self.minion_id, pillar, *val)
File "/opt/saltstack/salt/lib/python3.14/site-packages/salt/loader/lazy.py", line 177, in __call__
  ret = self.loader.run(run_func, *args, **kwargs)
File "/opt/saltstack/salt/lib/python3.14/site-packages/salt/loader/lazy.py", line 1480, in run
  return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs)
        ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/saltstack/salt/lib/python3.14/site-packages/salt/loader/lazy.py", line 1495, in _run_as
  ret = _func_or_method(*args, **kwargs)
File "/opt/saltstack/salt/lib/python3.14/site-packages/salt/pillar/git_pillar.py", line 449, in ext_pillar
  git_pillar = salt.utils.gitfs.GitPillar(
      opts,
  ...<3 lines>...
      global_only=GLOBAL_ONLY,
  )
File "/opt/saltstack/salt/lib/python3.14/site-packages/salt/utils/gitfs.py", line 3134, in __init__
  self.init_remotes(
  ~~~~~~~~~~~~~~~~~^
      remotes if remotes is not None else [],
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  ...<2 lines>...
      global_only,
      ^^^^^^^^^^^^
  )
  ^
File "/opt/saltstack/salt/lib/python3.14/site-packages/salt/utils/gitfs.py", line 3277, in init_remotes
  repo_obj = self.git_providers[self.provider](
      self.opts,
  ...<5 lines>...
      self.role,
  )
File "/opt/saltstack/salt/lib/python3.14/site-packages/salt/utils/gitfs.py", line 1795, in __init__
  super().__init__(
  ~~~~~~~~~~~~~~~~^
      opts,
      ^^^^^
  ...<5 lines>...
      role,
      ^^^^^
  )
  ^
File "/opt/saltstack/salt/lib/python3.14/site-packages/salt/utils/gitfs.py", line 531, in __init__
  self.fetch_request_check()
  ~~~~~~~~~~~~~~~~~~~~~~~~^^
File "/opt/saltstack/salt/lib/python3.14/site-packages/salt/utils/gitfs.py", line 1363, in fetch_request_check
  self.fetch()
  ~~~~~~~~~~^^
File "/opt/saltstack/salt/lib/python3.14/site-packages/salt/utils/gitfs.py", line 953, in fetch
  return self._fetch()
        ~~~~~~~~~~~^^
File "/opt/saltstack/salt/lib/python3.14/site-packages/salt/utils/gitfs.py", line 2188, in _fetch
  origin = self.repo.remotes[0]
          ^^^^^^^^^
What we've tried
  • Downgrading pygit2 (from 1.20.0 to 1.19.x) — no change, still intermittent.
  • Clearing the gitfs/git_pillar cache and git locks — gives a clean run or two, then it comes back.
  • Same setup/config running with salt-master version 3007.14 works fine
Type of salt install

Official rpm

Major version

3008.x

What supported OS are you seeing the problem on? Can select multiple. (If bug appears on an unsupported OS, please open a GitHub Discussion instead)

centos-stream-9

salt --versions-report output
Salt Version:
          Salt: 3008.2
 
Python Version:
        Python: 3.14.6 (main, Jun 11 2026, 02:19:05) [GCC 11.2.0]
 
Dependency Versions:
          cffi: 2.0.0
      cherrypy: 18.10.0
  cryptography: 48.0.0
      dateutil: 2.9.0.post0
     docker-py: Not Installed
         gitdb: 4.0.12
     gitpython: 3.1.50
        Jinja2: 3.1.6
       libgit2: 1.9.6
  looseversion: 1.3.0
      M2Crypto: Not Installed
          Mako: Not Installed
       msgpack: 1.1.2
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     packaging: 24.0
     pycparser: 3.00
      pycrypto: Not Installed
  pycryptodome: 3.23.0
        pygit2: 1.20.0
  python-gnupg: 0.5.6
        PyYAML: 6.0.3
         PyZMQ: 27.1.0
        relenv: 0.22.14
         smmap: 5.0.2
       timelib: 0.3.0
       Tornado: 6.5.7
           ZMQ: 4.3.5
 
Salt Package Information:
  Package Type: onedir
 
System Versions:
          dist: centos 9 
        locale: utf-8
       machine: x86_64
       release: 5.14.0-734.el9.x86_64
        system: Linux
       version: CentOS Stream 9

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with salt/utils/gitfs.py, especially GitFS fetch_request_check(), fetch(), and the Pygit2 path in _fetch(); then trace how salt/pillar/git_pillar.py constructs GitPillar. Reproduce by running highstate or state.apply repeatedly with pygit2 ext_pillar remotes, and consider the issue done when repeated runs consistently render successfully without the missing repo error.

Written by the indexing model from the issue text.

Assessment

Tech stack
git, python
Domain
devops, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.