New problem with pygit2 1.5.0 KeyError object not found <commit_id>
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 15.7k
- Forks
- 5.6k
- Avg merge
- 2d 44m
- Merged PRs (30d)
- 80
Description
Description of Issue
I have synchronization issues between a salt master server and gitfs through pygit2 module version 1.5.0 and with saltstack 3002.2.
Here is what I am seeing in my salt-master logs (after anonymization) :
journalctl -xn150 -f -u salt-master.service
[ERROR ] Exception caught while fetching gitfs remote 'ssh://git@host.domain.tld/repos/un-peu-de-sel.git': 'object not found - no match for id (3bc3ba574>
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/salt/utils/gitfs.py", line 2531, in fetch_remotes
if repo.fetch():
File "/usr/lib/python3/dist-packages/salt/utils/gitfs.py", line 820, in fetch
return self._fetch()
File "/usr/lib/python3/dist-packages/salt/utils/gitfs.py", line 1864, in _fetch
fetch_results = origin.fetch(**fetch_kwargs)
File "/usr/local/lib/python3.8/dist-packages/pygit2/remote.py", line 128, in fetch
payload.check_error(err)
File "/usr/local/lib/python3.8/dist-packages/pygit2/callbacks.py", line 93, in check_error
check_error(error_code)
File "/usr/local/lib/python3.8/dist-packages/pygit2/errors.py", line 56, in check_error
raise KeyError(message)
KeyError: 'object not found - no match for id (3bc3ba574e1992af38727fa67c3d6a541a2d1146)'
[ERROR ] Exception caught while fetching gitfs remote 'ssh://git@host.domain.tld/repos/modulefiles.git': 'object not found - no match for id (84fe4348c08ebd198>
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/salt/utils/gitfs.py", line 2531, in fetch_remotes
if repo.fetch():
File "/usr/lib/python3/dist-packages/salt/utils/gitfs.py", line 820, in fetch
return self._fetch()
File "/usr/lib/python3/dist-packages/salt/utils/gitfs.py", line 1864, in _fetch
fetch_results = origin.fetch(**fetch_kwargs)
File "/usr/local/lib/python3.8/dist-packages/pygit2/remote.py", line 128, in fetch
payload.check_error(err)
File "/usr/local/lib/python3.8/dist-packages/pygit2/callbacks.py", line 93, in check_error
check_error(error_code)
File "/usr/local/lib/python3.8/dist-packages/pygit2/errors.py", line 56, in check_error
raise KeyError(message)
KeyError: 'object not found - no match for id (84fe4348c08ebd198f85856077541bb5b4c97ebc)'
[ERROR ] Exception caught while fetching gitfs remote 'ssh://git@host.domain.tld/repos/un-peu-de-sel.git': 'object not found - no match for id (3bc3ba574>
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/salt/utils/gitfs.py", line 2531, in fetch_remotes
if repo.fetch():
File "/usr/lib/python3/dist-packages/salt/utils/gitfs.py", line 820, in fetch
return self._fetch()
File "/usr/lib/python3/dist-packages/salt/utils/gitfs.py", line 1864, in _fetch
fetch_results = origin.fetch(**fetch_kwargs)
File "/usr/local/lib/python3.8/dist-packages/pygit2/remote.py", line 128, in fetch
payload.check_error(err)
File "/usr/local/lib/python3.8/dist-packages/pygit2/callbacks.py", line 93, in check_error
check_error(error_code)
File "/usr/local/lib/python3.8/dist-packages/pygit2/errors.py", line 56, in check_error
raise KeyError(message)
KeyError: 'object not found - no match for id (3bc3ba574e1992af38727fa67c3d6a541a2d1146)'
[ERROR ] Exception caught while fetching gitfs remote 'ssh://git@host.domain.tld/repos/modulefiles.git': 'object not found - no match for id (84fe4348c08ebd198>
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/salt/utils/gitfs.py", line 2531, in fetch_remotes
if repo.fetch():
File "/usr/lib/python3/dist-packages/salt/utils/gitfs.py", line 820, in fetch
return self._fetch()
File "/usr/lib/python3/dist-packages/salt/utils/gitfs.py", line 1864, in _fetch
fetch_results = origin.fetch(**fetch_kwargs)
File "/usr/local/lib/python3.8/dist-packages/pygit2/remote.py", line 128, in fetch
payload.check_error(err)
File "/usr/local/lib/python3.8/dist-packages/pygit2/callbacks.py", line 93, in check_error
check_error(error_code)
File "/usr/local/lib/python3.8/dist-packages/pygit2/errors.py", line 56, in check_error
raise KeyError(message)
KeyError: 'object not found - no match for id (84fe4348c08ebd198f85856077541bb5b4c97ebc)'
Switching to a recent version of gitpython solved this issue.
Setup
My /etc/salt/master master configuration:
fileserver_backend:
- git
gitfs_provider: gitpython
gitfs_base: master
gitfs_remotes:
- ssh://git@host.domain.tld/repos/salt.git:
- name: states
- root: 'salt_states'
- pubkey: /root/.ssh/id_rsa.pub
- privkey: /root/.ssh/id_rsa
- ssh://git@host.domain.tld/repos/salt.git:
- name: reactor
- root: 'salt_reactor'
- pubkey: /root/.ssh/id_rsa.pub
- privkey: /root/.ssh/id_rsa
- ssh://git@host.domain.tld/repos/modulefiles.git:
- name: modulefiles
- pubkey: /root/.ssh/id_rsa.pub
- privkey: /root/.ssh/id_rsa
ext_pillar:
- git:
- master ssh://git@host.domain.tld/repos/salt.git:
- env: base
- name: pillar
- root: salt_pillar
- pubkey: /root/.ssh/id_rsa.pub
- privkey: /root/.ssh/id_rsa
Steps to Reproduce Issue
Install saltstack 3002.2 for python3, then install pygit2 1.5.0, and configure it to use a local git repository (I am using a local gitlab server with an autosign certificate. I tried to switch to a public server but I get the same error).
Versions Report
Salt Version:
Salt: 3002.2
Dependency Versions:
cffi: 1.14.3
cherrypy: Not Installed
dateutil: 2.7.3
docker-py: Not Installed
gitdb: 2.0.6
gitpython: 3.0.7
Jinja2: 2.10.1
libgit2: 1.1.0
M2Crypto: Not Installed
Mako: Not Installed
msgpack: 0.6.2
msgpack-pure: Not Installed
mysql-python: Not Installed
pycparser: 2.20
pycrypto: 2.6.1
pycryptodome: 3.6.1
pygit2: 1.5.0
Python: 3.8.5 (default, Jul 28 2020, 12:59:40)
python-gnupg: 0.4.5
PyYAML: 5.3.1
PyZMQ: 18.1.1
smmap: 2.0.5
timelib: Not Installed
Tornado: 4.5.3
ZMQ: 4.3.2
System Versions:
dist: ubuntu 20.04 focal
locale: utf-8
machine: x86_64
release: 5.4.0-54-generic
system: Linux
version: Ubuntu 20.04 focal
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in salt/utils/gitfs.py at fetch_remotes, fetch, and _fetch, then compare the pygit2 and GitPython paths described in the traceback. Reproduce with Salt 3002.2, pygit2 1.5.0, and the listed gitfs configuration; done means fetching the configured remotes no longer raises the reported KeyError.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- git, python
- Domain
- devops, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100