[BUG] Git module and states are broken in multiple ways
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 15.7k
- Forks
- 5.6k
- Avg merge
- 2d 44m
- Merged PRs (30d)
- 80
Description
Description
- git state fails if rev is SHA1
- logging is uninformative unless you set GIT_SSH_COMMAND="ssh -v " environ
- (pretty sure, but not completely) GIT_IDENTITY env of git is nowhere to be found, but is used here. I'm pretty sure it's useless and must be rewtitten to GIT_SSH_COMMAND="ssh -i "
First of all - without GIT_SSH_COMMAND="ssh -v" (see states below) logging is useless. You would always receive this:
12:43:54 [INFO ] File /root/.ssh/repo-sync-key is in the correct state [salt.state]
12:43:54 [INFO ] Completed state [/root/.ssh/repo-sync-key] at time 12:43:54.973777 (duration_in_ms=32.855) [salt.state]
12:43:54 [INFO ] Running state [git+ssh://git@<local mirror URL>/gentoo-mirror] at time 12:43:54.974039 [salt.state]
12:43:54 [INFO ] Executing state git.latest for [git+ssh://git@<local mirror URL>/gentoo-mirror] [salt.state]
12:43:54 [INFO ] Executing command ['git', 'config', '--global', '--get-regexp', 'filter\\.lfs\\.'] in directory '/root' [salt.loaded.int.module.cmdmod]
12:43:54 [DEBUG ] retcode: 1 [salt.loaded.int.module.cmdmod]
12:43:54 [INFO ] Checking remote revision for git+ssh://git@<local mirror URL>/gentoo-mirror [salt.loaded.int.states.git]
12:43:54 [INFO ] Attempting git authentication using identity file /root/.ssh/repo-sync-key [salt.loaded.int.module.git]
12:43:54 [INFO ] Executing command ['git', 'ls-remote', 'git+ssh://git@<local mirror URL>/gentoo-mirror'] in directory '/root' [salt.loaded.int.module.cmdmod]
12:43:55 [ERROR ] Command '['git', 'ls-remote', 'git+ssh://git@<local mirror URL>/gentoo-mirror']' failed with return code: 128 [salt.loaded.int.module.cmdmod]
12:43:55 [ERROR ] stderr: git@<local mirror URL>: Permission denied (keyboard-interactive).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists. [salt.loaded.int.module.cmdmod]
I know this because at first I missed part of the docs that says you must not set depth parameter when setting rev to SHA1 instead of branch/tag. But kept getting the same Permission denied error and was looking in wrong direction.
So after a bit of digging into the modules/git.py and states/git.py I found out that module is using unfamiliar GIT_IDENTITY to set private key path. I desided to explicitly set GIT_SSH_COMMAND="ssh -vvv -i /root/.ssh/repo-sync-key" via another state and voila! The error message became informative and I saw this error.
And the last thing about GIT_IDENTITY - it turned out that I had .ssh/config with IdentityFile /root/.ssh/repo-sync-key set there all the time (so -i /root/.ssh/repo-sync-key earlier was excessive and it was -vvv part that mattered - I guess I was lucky to catch the problem at all). I removed IdentityFile from ssh config, -i part from GIT_SSH_COMMAND, but kept identity definition in the state and received the same Permission denied error I was getting earlier. From here I deduced that GIT_IDENTITY env is pretty much useless and identity feature in git.latest state is broken.
To clarify - the private key and .ssh have all the nesessary permissions and I'm able to work with the repo directly with git from OS.
And last, but not least - when you set rev to SHA1 you get this:
----------
ID: gentoo-repo
Function: git.latest
Name: git+ssh://git@<local mirror URL>/gentoo-mirror
Result: False
Comment: Fetch did not successfully retrieve rev '7b901fbb06' from git+ssh://git@<local mirror URL>/gentoo-mirror: Command 'git rev-parse 7b901fbb06^{commit}' failed: fatal: ambiguous argument '7b901fbb06^{commit}': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
Changes:
The exception is catched here. It's obvious for me that we should check if ^{commit} should be appended to the revision before invoking git rev-parse.
Setup
To reproduce you would need a git repository that works over git+ssh with publickey auth.
This is the state file to reproduce issue 1. git state fails if rev is SHA1
>>> gentoo/repos/gentoo.sls
include:
- .keys # sync .ssh/config and .ssh/repo-sync-key
setenv:
environ.setenv:
- name: GIT_SSH_COMMAND
- value: "ssh -v"
gentoo-repo:
git.latest:
- name: git+ssh://git@<local mirror URL>/gentoo-mirror
- target: /usr/portage
- rev: 7b901fbb06 # <---------- the only relevant thing here
- identity: /root/.ssh/repo-sync-key
- force_clone: True
- force_checkout: True
- reload_modules: True
- require:
- file: /root/.ssh/repo-sync-key
- environ: setenv
To reprodure 2. logging is uninformative unless you set GIT_SSH_COMMAND="ssh -v " environ remove setenv state.
To reproduce 3. GIT_IDENTITY env of git is nowhere to be found remove .ssh/config or comment out IdentityFile part.
Steps to Reproduce the behavior
Run salt-call state.sls gentoo.repos.gentoo in each of 3 cases.
Expected behavior
- Provided SHA1 commit is checked out when
rev: <SHA1>is present in the state definition. - Logging is informative.
- Authentication is successful when path to correct identity file is provided in the state definition.
Versions Report
salt --versions-report
On minion: ``` Salt Version: Salt: 3002.2Dependency Versions:
cffi: 1.14.4
cherrypy: Not Installed
dateutil: 2.8.1
docker-py: Not Installed
gitdb: Not Installed
gitpython: Not Installed
Jinja2: 2.11.3
libgit2: Not Installed
M2Crypto: Not Installed
Mako: Not Installed
msgpack: 1.0.2
msgpack-pure: Not Installed
mysql-python: Not Installed
pycparser: 2.20
pycrypto: 3.9.9
pycryptodome: 3.9.9
pygit2: Not Installed
Python: 3.8.7 (default, Jan 25 2021, 13:32:46)
python-gnupg: 0.4.6
PyYAML: 5.4.1
PyZMQ: 19.0.1
smmap: Not Installed
timelib: Not Installed
Tornado: 4.5.3
ZMQ: 4.3.3
System Versions:
dist: gentoo 2.7
locale: utf-8
machine: x86_64
release: 5.8.18
system: Linux
version: Gentoo 2.7
On master:
Salt Version:
Salt: 3000.2
Dependency Versions:
cffi: 1.14.0
cherrypy: Not Installed
dateutil: 2.8.1
docker-py: Not Installed
gitdb: Not Installed
gitpython: Not Installed
Jinja2: 2.11.2
libgit2: Not Installed
M2Crypto: Not Installed
Mako: Not Installed
msgpack-pure: Not Installed
msgpack-python: 0.6.2
mysql-python: Not Installed
pycparser: 2.20
pycrypto: 3.9.8
pycryptodome: Not Installed
pygit2: Not Installed
Python: 3.7.8 (default, Jul 28 2020, 21:12:08)
python-gnupg: 0.4.6
PyYAML: 5.3.1
PyZMQ: 19.0.1
smmap: Not Installed
timelib: Not Installed
Tornado: 4.5.3
ZMQ: 4.3.2
System Versions:
dist: gentoo 2.6
locale: UTF-8
machine: x86_64
release: 5.4.92-gentoo-xen-guest-cgroups
system: Linux
version: Gentoo Base System 2.6
</details>
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the three cases with salt-call state.sls gentoo.repos.gentoo using the provided state file and version details. Read the referenced logic in salt/modules/git.py and salt/states/git.py, especially the linked identity handling and rev-parse paths. Done means SHA1 revisions check out, SSH diagnostics are informative, and the state’s identity path authenticates successfully.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- git, python
- Domain
- devops, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100