gitfs backend with root/mountpoint seems doesn't work with Windows
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 15.7k
- Forks
- 5.6k
- Avg merge
- 2d 44m
- Merged PRs (30d)
- 80
Description
Description of Issue
I arrive to connect remotly on my git repo with the backend gitfs.
But when I try to change root, it seems doesn't work (Windows).
I edit file C:\salt\bin\Lib\site-packages\salt\utils\gitfs.py and I add on line 2348 (version 2019.2) the following code:
from pprint import pprint
pprint(repo.file_list('salt'))
And I have the following output without root:
({'.vscode/settings.json',
'LICENSE',
'README.md',
'salt/xxx/defaults.yaml',
'salt/xxx/init.sls',
'top.sls'},
{})
And with root:
({'xxx\\defaults.yaml',
'xxx\\init.sls'},
{})
First has a slash and the second has a backslash. I think that's why I have the issue.
Setup
Note I have the same issue with pygit2 and gitpython.
My custom config:
file_client: local
fileserver_backend:
- roots
- git
top_file_merging_strategy: same
gitfs_provider: pygit2
# gitfs_provider: gitpython
gitfs_base: salt
gitfs_remotes:
- https://xxx.git:
- root: salt
- mountpoint: salt://
- base: salt
Steps to Reproduce Issue
Without options root and mountpoint activated return True:
PS C:\WINDOWS\system32> C:\salt\bin\python.exe C:\salt\bin\Scripts\salt-call state.sls_exists salt/xxx
[ERROR ] Exception caught while fetching gitfs remote 'https://xxx.git': environment can only contain strings
Traceback (most recent call last):
File "C:\salt\bin\lib\site-packages\salt\utils\gitfs.py", line 2348, in fetch_remotes
if repo.fetch():
File "C:\salt\bin\lib\site-packages\salt\utils\gitfs.py", line 767, in fetch
return self._fetch()
File "C:\salt\bin\lib\site-packages\salt\utils\gitfs.py", line 1767, in _fetch
cleaned = self.clean_stale_refs(local_refs=refs_post)
File "C:\salt\bin\lib\site-packages\salt\utils\gitfs.py", line 1623, in clean_stale_refs
return super(Pygit2, self).clean_stale_refs()
File "C:\salt\bin\lib\site-packages\salt\utils\gitfs.py", line 602, in clean_stale_refs
stderr=subprocess.STDOUT)
File "C:\salt\bin\lib\subprocess.py", line 676, in __init__
restore_signals, start_new_session)
File "C:\salt\bin\lib\subprocess.py", line 957, in _execute_child
startupinfo)
TypeError: environment can only contain strings
local:
True
With options root and mountpoint activated return False:
PS C:\WINDOWS\system32> C:\salt\bin\python.exe C:\salt\bin\Scripts\salt-call state.sls_exists xxx
[ERROR ] Exception caught while fetching gitfs remote 'https://xxx.git': environment can only contain strings
Traceback (most recent call last):
File "C:\salt\bin\lib\site-packages\salt\utils\gitfs.py", line 2348, in fetch_remotes
if repo.fetch():
File "C:\salt\bin\lib\site-packages\salt\utils\gitfs.py", line 767, in fetch
return self._fetch()
File "C:\salt\bin\lib\site-packages\salt\utils\gitfs.py", line 1767, in _fetch
cleaned = self.clean_stale_refs(local_refs=refs_post)
File "C:\salt\bin\lib\site-packages\salt\utils\gitfs.py", line 1623, in clean_stale_refs
return super(Pygit2, self).clean_stale_refs()
File "C:\salt\bin\lib\site-packages\salt\utils\gitfs.py", line 602, in clean_stale_refs
stderr=subprocess.STDOUT)
File "C:\salt\bin\lib\subprocess.py", line 676, in __init__
restore_signals, start_new_session)
File "C:\salt\bin\lib\subprocess.py", line 957, in _execute_child
startupinfo)
TypeError: environment can only contain strings
local:
False
Also note and I don't know why (I haven't debug) that repo.fetch return an error !
Versions Report
PS C:\WINDOWS\system32> C:\salt\bin\python.exe C:\salt\bin\Scripts\salt-call --versions-report
Salt Version:
Salt: 2019.2.0
Dependency Versions:
cffi: 1.11.5
cherrypy: 17.4.1
dateutil: 2.7.5
docker-py: Not Installed
gitdb: 2.0.5
gitpython: 2.1.10
ioflo: Not Installed
Jinja2: 2.10
libgit2: 0.28.2
libnacl: 1.6.1
M2Crypto: Not Installed
Mako: 1.0.7
msgpack-pure: Not Installed
msgpack-python: 0.5.6
mysql-python: Not Installed
pycparser: 2.17
pycrypto: 2.6.1
pycryptodome: Not Installed
pygit2: 0.28.2
Python: 3.5.4 (v3.5.4:3f56838, Aug 8 2017, 02:17:05) [MSC v.1900 64 bit (AMD64)]
python-gnupg: 0.4.3
PyYAML: 3.13
PyZMQ: 17.1.2
RAET: Not Installed
smmap: 2.0.5
timelib: 0.2.4
Tornado: 4.5.3
ZMQ: 4.2.5
System Versions:
dist:
locale: cp1252
machine: AMD64
release: 10
system: Windows
version: 10 10.0.17134 SP0 Multiprocessor Free
The main issue is the root and mountpoint options and the minor is the traceback TypeError: environment can only contain strings
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in salt/utils/gitfs.py at the reported lines 602, 767, 1623, 1767, and 2348, then reproduce the Windows configuration with root and mountpoint enabled. Compare the file paths returned with and without root and investigate the reported fetch traceback. Done means the configured root and mountpoint work on Windows and the environment TypeError is resolved.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100