saltstack / saltstack/salt

[BUG] gitfs fileserver does not sync to all_saltenvs for remote

Open
#62,077 7 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug File Servers gitfs needs-triage
Dominant language
Python
Stars
15.7k
Forks
5.6k
Avg merge
2d 44m
Merged PRs (30d)
80

Description

Description
When utilizing the all_saltenvs option for a gitfs remote resource, you would expect that the resource would be synced to all environments identified by gitfs specified by file_roots in the master configuration. The resource however is only synced to the base environment.

Setup
The following is the master configuration:

git_pillar_provider: pygit2

fileserver_backend:
  - roots
  - git

gitfs_remotes:
  - https://github.com/saltstack-formulas/vim-formula:
    - all_saltenvs: v0.15.5
    - root: vim
    - mountpoint: salt://vim

file_roots:
  base:
    - /srv/salt/states
  dev:
    - /srv/salt/states

pillar_roots:
  base:
    - /srv/salt/pillar/base
  dev:
    - /srv/salt/pillar/dev
    - /srv/salt/pillar/base

pillar_source_merging_strategy: recurse
pillarenv_from_saltenv: True
top_file_merging_strategy: same

The following is the additional minion configuration:

pillarenv: dev
saltenv: dev

The following is the directory structure of /srv/salt:

[root@lima-salt-master salt]# pwd
/srv/salt
[root@lima-salt-master salt]# tree
.
├── formulas
├── pillar
│   ├── base
│   │   ├── hello
│   │   │   └── world.sls
│   │   ├── nginx
│   │   │   └── init.sls
│   │   └── top.sls -> /srv/salt/pillar/top.sls
│   ├── dev
│   │   └── top.sls -> /srv/salt/pillar/top.sls
│   └── top.sls
└── states
    └── top.sls

7 directories, 6 files

The contents of the states/top.sls:

{{ saltenv }}:
  '*':
    - vim

Please be as specific as possible and give set-up details.

  • [X ] VM (lima virtualized instance of Oracle Linux 8 on a MacOS host)

Steps to Reproduce the behavior
The above configuration syncs the vim formula resource to the base environment correctly:

[root@lima-salt-master salt]# salt-run fileserver.file_list
- top.sls
- vim/_mapdata/_mapdata.jinja
- vim/_mapdata/init.sls
- vim/absent.sls
- vim/editor.sls
- vim/files/nerdtree/doc/NERD_tree.txt
- vim/files/nerdtree/nerdtree_plugin/exec_menuitem.vim
- vim/files/nerdtree/nerdtree_plugin/fs_menu.vim
- vim/files/nerdtree/plugin/NERD_tree.vim
- vim/files/nerdtree/syntax/nerdtree.vim
- vim/files/pyflakes/ftplugin/python/pyflakes.vim
- vim/files/pyflakes/ftplugin/python/pyflakes/LICENSE
- vim/files/pyflakes/ftplugin/python/pyflakes/NEWS.txt
- vim/files/pyflakes/ftplugin/python/pyflakes/README.rst
- vim/files/pyflakes/ftplugin/python/pyflakes/bin/pyflakes
- vim/files/pyflakes/ftplugin/python/pyflakes/pyflakes/__init__.py
- vim/files/pyflakes/ftplugin/python/pyflakes/pyflakes/checker.py
- vim/files/pyflakes/ftplugin/python/pyflakes/pyflakes/messages.py
- vim/files/pyflakes/ftplugin/python/pyflakes/pyflakes/scripts/__init__.py
- vim/files/pyflakes/ftplugin/python/pyflakes/pyflakes/scripts/pyflakes.py
- vim/files/pyflakes/ftplugin/python/pyflakes/pyflakes/test/__init__.py
- vim/files/pyflakes/ftplugin/python/pyflakes/pyflakes/test/harness.py
- vim/files/pyflakes/ftplugin/python/pyflakes/pyflakes/test/test_imports.py
- vim/files/pyflakes/ftplugin/python/pyflakes/pyflakes/test/test_other.py
- vim/files/pyflakes/ftplugin/python/pyflakes/pyflakes/test/test_script.py
- vim/files/pyflakes/ftplugin/python/pyflakes/pyflakes/test/test_undefined_names.py
- vim/files/pyflakes/ftplugin/python/pyflakes/setup.py
- vim/files/salt/ftdetect/jinja.vim
- vim/files/salt/ftdetect/sls.vim
- vim/files/salt/ftplugin/rst.vim
- vim/files/salt/ftplugin/sls.vim
- vim/files/salt/syntax/jinja.vim
- vim/files/salt/syntax/sls.vim
- vim/files/vimrc
- vim/init.sls
- vim/map.jinja
- vim/nerdtree.sls
- vim/pyflakes.sls
- vim/salt.sls

It does not sync the resource to the dev environment:

[root@lima-salt-master salt]# salt-run fileserver.update
True
[root@lima-salt-master salt]# salt-run fileserver.file_list saltenv=dev
- top.sls

Additional list of the environments on the fileserver:

[root@lima-salt-master salt]# salt-run fileserver.envs
- base
- dev
- v0.14.2
- v0.15.0
- v0.15.1
- v0.15.2
- v0.15.3
- v0.15.4
- v0.15.5

Expected behavior
I would expect that the vim formula also be present within the dev environment.

Versions Report

salt --versions-report (Provided by running salt --versions-report. Please also mention any differences in master/minion versions.)
Salt Version:
          Salt: 3004.1

Dependency Versions:
          cffi: 1.11.5
      cherrypy: unknown
      dateutil: 2.6.1
     docker-py: Not Installed
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 2.10.1
       libgit2: 0.26.8
      M2Crypto: 0.35.2
          Mako: Not Installed
       msgpack: 0.6.2
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     pycparser: 2.14
      pycrypto: Not Installed
  pycryptodome: Not Installed
        pygit2: 0.26.4
        Python: 3.6.8 (default, Nov 10 2021, 06:50:23)
  python-gnupg: Not Installed
        PyYAML: 3.12
         PyZMQ: 19.0.0
         smmap: Not Installed
       timelib: Not Installed
       Tornado: 4.5.3
           ZMQ: 4.3.4

System Versions:
          dist: oracle 8.5
        locale: UTF-8
       machine: x86_64
       release: 5.4.17-2136.302.7.2.2.el8uek.x86_64
        system: Linux
       version: Oracle Linux Server 8.5

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the issue with the shown master and minion configuration, then compare salt-run fileserver.file_list for the base and dev environments after salt-run fileserver.update. Use states/top.sls, the gitfs_remotes settings, and fileserver.envs output to trace the environment mapping. Done means the vim formula appears in the dev file list as well as base.

Written by the indexing model from the issue text.

Assessment

Tech stack
git, python
Domain
devops, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.