saltstack / saltstack/salt

[BUG] file_ignore_regex and file_ignore_glob not applied for gitfs remote

Open
#62,368 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug File Servers needs-triage
Dominant language
Python
Stars
15.7k
Forks
5.6k
Avg merge
2d 44m
Merged PRs (30d)
80

Description

Description
The options file_ignore_regex and file_ignore_glob in master configuration are ignored for gitfs remotes (on GitHub in this case).

Setup
This master setup has files in gitfs, locally, and in winrepo.

fileserver_backend:
  - roots
  - gitfs
 
# file_roots is not defined, so default is used. Adding it for fun made no difference.

gitfs_provider: pygit2
gitfs_update_interval: 120

# (FOR CONTEXT WE HAVE MORE THAN ONE REMOTE)
gitfs_remotes:
  - git@github.com:mycompany/my-formulas.git:
    - mountpoint: salt://formulas
    - pubkey: /path/to/key_rsa.pub
    - privkey: /path/to/key_rsa

winrepo_remotes: []

winrepo_dir_ng: '/srv/salt/win/repo-ng'

winrepo_remotes_ng:
  - master git@github.com:mycompany/saltstack-winrepo.git:
    - pubkey: /path/to/key_rsa.pub
    - privkey: /path/to/key_rsa

Test case is to ignore README.md files for the fileserver.

file_ignore_regex:
  - '/README.md$'

OR

file_ignore_glob:
  - '*.md'

I tested both cases one at a time and both at the same time.

Steps to Reproduce the behavior

Without ignoring README.md:

# salt-run fileserver.file_list | grep README
- automation-dev-dee/orchestrate/README.md
- formulas/README.md
- win/repo-ng/saltstack-winrepo/README.md
  • automation-dev-dee = files on local master under /srv/salt/
  • formulas = gitfs remote on GitHub
  • win = winrepo files (as configured above)

Using file_ignore_regex and file_ignore_glob to ignore README.md files (one at a time or both) doesn't ignore any of gitfs remotes

# salt-run fileserver.file_list | grep README
- formulas/README.md

Expected behavior

All targeted files are ignored.

# salt-run fileserver.file_list | grep README

Versions Report

Salt Version:
          Salt: 3004.2
 
Dependency Versions:
          cffi: 1.11.5
      cherrypy: Not Installed
      dateutil: 2.6.1
     docker-py: Not Installed
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 2.10.1
       libgit2: 0.28.4
      M2Crypto: 0.35.2
          Mako: Not Installed
       msgpack: 0.6.2
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     pycparser: 2.14
      pycrypto: Not Installed
  pycryptodome: Not Installed
        pygit2: 1.0.3
        Python: 3.6.8 (default, Jan 14 2022, 11:04:20)
  python-gnupg: Not Installed
        PyYAML: 3.12
         PyZMQ: 19.0.0
         smmap: Not Installed
       timelib: Not Installed
       Tornado: 4.5.3
           ZMQ: 4.3.4
 
System Versions:
          dist: rhel 8.6 Ootpa
        locale: UTF-8
       machine: x86_64
       release: 4.18.0-372.9.1.el8.x86_64
        system: Linux
       version: Red Hat Enterprise Linux 8.6 Ootpa

Thanks!

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the salt-run fileserver.file_list entry point and trace how file_ignore_regex and file_ignore_glob are applied to local files, winrepo, and gitfs remotes. Done means README.md files are excluded from gitfs output as they are from the other configured sources, with regression coverage for both options.

Written by the indexing model from the issue text.

Assessment

Tech stack
git, python
Domain
infrastructure
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.