saltstack / saltstack/salt

[BUG] GitFS hang until longest update_interval

Open
#64,615 3 comments 0 reactions 1 assignee View on GitHub

@dwoz is already working on this.

Since Jul 7, 2023.

bug gitfs needs-triage VMware
Dominant language
Python
Stars
15.7k
Forks
5.6k
Avg merge
2d 44m
Merged PRs (30d)
80

Description

Description
GitFS will hang after a certain amount of time. On my machine, it was about 5 minutes. I don't know if it's based on time or the number of refreshes, which on my machine was about 10 refreshes. It will do no more fetches until the last defined GitFS remote update_interval fires. It is then issued a new PID and the cycle starts over. This would not be noticeable if your longest update_interval is close to the other update intervals. But if you have an update interval of a day (86400), then your GitFS remotes will update for 5 minutes every day.

Setup
master config

loop_interval: 15

log_level_logfile: all

gitfs_disable_saltenv_mapping: True
gitfs_provider: pygit2
gitfs_remotes:
  - https://github.com/saltstack/salt-bootstrap.git:
    - base: stable
    - mountpoint: salt://bootstrap
    - update_interval: 360
  - https://github.com/DataDog/datadog-formula.git:
    - base: 3.4
    - mountpoint: salt://datadog
    - root: datadog
    - update_interval: 30
fileserver_backend:
  - roots
  - gitfs
# Files on GitHub to ignore on the SaltStack fileserver
file_ignore_regex:
  - '/\.git($|/)'
  - '/\.gitignore'
  - '/\.pre-commit-config.yaml'

Steps to Reproduce the behavior
Start the master:

sudo salt-master -l debug

In another terminal on the same machine, tail the master log grepping for gitfs:

sudo tail -f /var/log/salt/master | grep gitfs

You will see the following output:

[shane@localhost ~]$ sudo tail -f /var/log/salt/master | grep gitfs
[sudo] password for shane:

>>>>>>>> Notice that the refresh occurs every 30 seconds as expected

2023-07-07 15:51:38,602 [salt.master      :450 ][DEBUG   ][171559] Updating gitfs fileserver cache for the following targets: [('https://github.com/DataDog/datadog-formula.git', None)]
2023-07-07 15:51:38,602 [salt.utils.gitfs :2885][DEBUG   ][171559] Re-using gitfs object for process 171559
2023-07-07 15:51:38,603 [salt.utils.gitfs :919 ][DEBUG   ][171559] Set update lock for gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 15:51:38,604 [salt.utils.gitfs :817 ][DEBUG   ][171559] Fetching gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 15:51:38,956 [salt.utils.gitfs :1917][DEBUG   ][171559] gitfs remote 'https://github.com/DataDog/datadog-formula.git' is up-to-date
2023-07-07 15:51:38,958 [salt.utils.gitfs :697 ][DEBUG   ][171559] Removed update lock for gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 15:52:08,960 [salt.master      :450 ][DEBUG   ][171559] Updating gitfs fileserver cache for the following targets: [('https://github.com/DataDog/datadog-formula.git', None)]
2023-07-07 15:52:08,961 [salt.utils.gitfs :2885][DEBUG   ][171559] Re-using gitfs object for process 171559
2023-07-07 15:52:08,961 [salt.utils.gitfs :919 ][DEBUG   ][171559] Set update lock for gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 15:52:08,962 [salt.utils.gitfs :817 ][DEBUG   ][171559] Fetching gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 15:52:09,323 [salt.utils.gitfs :1917][DEBUG   ][171559] gitfs remote 'https://github.com/DataDog/datadog-formula.git' is up-to-date
2023-07-07 15:52:09,326 [salt.utils.gitfs :697 ][DEBUG   ][171559] Removed update lock for gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 15:52:39,327 [salt.master      :450 ][DEBUG   ][171559] Updating gitfs fileserver cache for the following targets: [('https://github.com/DataDog/datadog-formula.git', None)]
2023-07-07 15:52:39,328 [salt.utils.gitfs :2885][DEBUG   ][171559] Re-using gitfs object for process 171559
2023-07-07 15:52:39,328 [salt.utils.gitfs :919 ][DEBUG   ][171559] Set update lock for gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 15:52:39,328 [salt.utils.gitfs :817 ][DEBUG   ][171559] Fetching gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 15:52:39,668 [salt.utils.gitfs :1917][DEBUG   ][171559] gitfs remote 'https://github.com/DataDog/datadog-formula.git' is up-to-date
2023-07-07 15:52:39,671 [salt.utils.gitfs :697 ][DEBUG   ][171559] Removed update lock for gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 15:53:09,672 [salt.master      :450 ][DEBUG   ][171559] Updating gitfs fileserver cache for the following targets: [('https://github.com/DataDog/datadog-formula.git', None)]
2023-07-07 15:53:09,673 [salt.utils.gitfs :2885][DEBUG   ][171559] Re-using gitfs object for process 171559
2023-07-07 15:53:09,674 [salt.utils.gitfs :919 ][DEBUG   ][171559] Set update lock for gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 15:53:09,674 [salt.utils.gitfs :817 ][DEBUG   ][171559] Fetching gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 15:53:10,010 [salt.utils.gitfs :1917][DEBUG   ][171559] gitfs remote 'https://github.com/DataDog/datadog-formula.git' is up-to-date
2023-07-07 15:53:10,012 [salt.utils.gitfs :697 ][DEBUG   ][171559] Removed update lock for gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 15:53:40,014 [salt.master      :450 ][DEBUG   ][171559] Updating gitfs fileserver cache for the following targets: [('https://github.com/DataDog/datadog-formula.git', None)]
2023-07-07 15:53:40,015 [salt.utils.gitfs :2885][DEBUG   ][171559] Re-using gitfs object for process 171559
2023-07-07 15:53:40,016 [salt.utils.gitfs :919 ][DEBUG   ][171559] Set update lock for gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 15:53:40,017 [salt.utils.gitfs :817 ][DEBUG   ][171559] Fetching gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 15:53:40,354 [salt.utils.gitfs :1917][DEBUG   ][171559] gitfs remote 'https://github.com/DataDog/datadog-formula.git' is up-to-date
2023-07-07 15:53:40,355 [salt.utils.gitfs :697 ][DEBUG   ][171559] Removed update lock for gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 15:54:10,357 [salt.master      :450 ][DEBUG   ][171559] Updating gitfs fileserver cache for the following targets: [('https://github.com/DataDog/datadog-formula.git', None)]
2023-07-07 15:54:10,357 [salt.utils.gitfs :2885][DEBUG   ][171559] Re-using gitfs object for process 171559
2023-07-07 15:54:10,358 [salt.utils.gitfs :919 ][DEBUG   ][171559] Set update lock for gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 15:54:10,359 [salt.utils.gitfs :817 ][DEBUG   ][171559] Fetching gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 15:54:10,732 [salt.utils.gitfs :1917][DEBUG   ][171559] gitfs remote 'https://github.com/DataDog/datadog-formula.git' is up-to-date
2023-07-07 15:54:10,735 [salt.utils.gitfs :697 ][DEBUG   ][171559] Removed update lock for gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 15:54:40,737 [salt.master      :450 ][DEBUG   ][171559] Updating gitfs fileserver cache for the following targets: [('https://github.com/DataDog/datadog-formula.git', None)]
2023-07-07 15:54:40,737 [salt.utils.gitfs :2885][DEBUG   ][171559] Re-using gitfs object for process 171559
2023-07-07 15:54:40,739 [salt.utils.gitfs :919 ][DEBUG   ][171559] Set update lock for gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 15:54:40,739 [salt.utils.gitfs :817 ][DEBUG   ][171559] Fetching gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 15:54:41,091 [salt.utils.gitfs :1917][DEBUG   ][171559] gitfs remote 'https://github.com/DataDog/datadog-formula.git' is up-to-date
2023-07-07 15:54:41,094 [salt.utils.gitfs :697 ][DEBUG   ][171559] Removed update lock for gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 15:55:11,096 [salt.master      :450 ][DEBUG   ][171559] Updating gitfs fileserver cache for the following targets: [('https://github.com/DataDog/datadog-formula.git', None)]
2023-07-07 15:55:11,097 [salt.utils.gitfs :2885][DEBUG   ][171559] Re-using gitfs object for process 171559
2023-07-07 15:55:11,098 [salt.utils.gitfs :919 ][DEBUG   ][171559] Set update lock for gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 15:55:11,098 [salt.utils.gitfs :817 ][DEBUG   ][171559] Fetching gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 15:55:11,431 [salt.utils.gitfs :1917][DEBUG   ][171559] gitfs remote 'https://github.com/DataDog/datadog-formula.git' is up-to-date
2023-07-07 15:55:11,432 [salt.utils.gitfs :697 ][DEBUG   ][171559] Removed update lock for gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 15:55:41,433 [salt.master      :450 ][DEBUG   ][171559] Updating gitfs fileserver cache for the following targets: [('https://github.com/DataDog/datadog-formula.git', None)]
2023-07-07 15:55:41,434 [salt.utils.gitfs :2885][DEBUG   ][171559] Re-using gitfs object for process 171559
2023-07-07 15:55:41,435 [salt.utils.gitfs :919 ][DEBUG   ][171559] Set update lock for gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 15:55:41,435 [salt.utils.gitfs :817 ][DEBUG   ][171559] Fetching gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 15:55:41,794 [salt.utils.gitfs :1917][DEBUG   ][171559] gitfs remote 'https://github.com/DataDog/datadog-formula.git' is up-to-date
2023-07-07 15:55:41,796 [salt.utils.gitfs :697 ][DEBUG   ][171559] Removed update lock for gitfs remote 'https://github.com/DataDog/datadog-formula.git'

>>>>>>>> 30-second refreshes stop here for 1.5 minutes, then the PID changes from 171559 to 171362

2023-07-07 15:57:18,335 [salt.utils.gitfs :2775][DEBUG   ][171362] pygit2 gitfs_provider enabled
2023-07-07 15:57:18,335 [salt.utils.gitfs :2879][DEBUG   ][171362] Created gitfs object with uninitialized remotes
2023-07-07 15:57:18,335 [salt.loader.lazy :963 ][DEBUG   ][171362] The functions from module 'gitfs' are being loaded by dir() on the loaded module
2023-07-07 15:57:18,335 [salt.utils.lazy  :99  ][DEBUG   ][171362] LazyLoaded gitfs.envs
2023-07-07 15:57:18,343 [salt.utils.gitfs :2885][DEBUG   ][171362] Re-using gitfs object for process 171362

>>>>>>>>>> Now it will refresh the bootstrap, and expected behavior resumes
>>>>>>>>>> With an interesting PID change again, from 171362 to 176464
>>>>>>>>>> You can see this happen one more time during this log dump, just watch the PID changes

2023-07-07 15:57:18,378 [salt.master      :450 ][DEBUG   ][176464] Updating gitfs fileserver cache for the following targets: [('https://github.com/saltstack/salt-bootstrap.git', None)]
2023-07-07 15:57:18,379 [salt.utils.gitfs :2885][DEBUG   ][176464] Re-using gitfs object for process 176464
2023-07-07 15:57:18,382 [salt.master      :450 ][DEBUG   ][176464] Updating gitfs fileserver cache for the following targets: [('https://github.com/DataDog/datadog-formula.git', None)]
2023-07-07 15:57:18,382 [salt.utils.gitfs :2885][DEBUG   ][176464] Re-using gitfs object for process 176464
2023-07-07 15:57:18,387 [salt.utils.gitfs :919 ][DEBUG   ][176464] Set update lock for gitfs remote 'https://github.com/saltstack/salt-bootstrap.git'
2023-07-07 15:57:18,388 [salt.utils.gitfs :817 ][DEBUG   ][176464] Fetching gitfs remote 'https://github.com/saltstack/salt-bootstrap.git'
2023-07-07 15:57:18,389 [salt.utils.gitfs :919 ][DEBUG   ][176464] Set update lock for gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 15:57:18,393 [salt.utils.gitfs :817 ][DEBUG   ][176464] Fetching gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 15:57:18,732 [salt.utils.gitfs :1917][DEBUG   ][176464] gitfs remote 'https://github.com/DataDog/datadog-formula.git' is up-to-date
2023-07-07 15:57:18,735 [salt.utils.gitfs :697 ][DEBUG   ][176464] Removed update lock for gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 15:57:18,895 [salt.utils.gitfs :1917][DEBUG   ][176464] gitfs remote 'https://github.com/saltstack/salt-bootstrap.git' is up-to-date
2023-07-07 15:57:18,900 [salt.utils.gitfs :697 ][DEBUG   ][176464] Removed update lock for gitfs remote 'https://github.com/saltstack/salt-bootstrap.git'
2023-07-07 15:57:48,737 [salt.master      :450 ][DEBUG   ][176464] Updating gitfs fileserver cache for the following targets: [('https://github.com/DataDog/datadog-formula.git', None)]
2023-07-07 15:57:48,737 [salt.utils.gitfs :2885][DEBUG   ][176464] Re-using gitfs object for process 176464
2023-07-07 15:57:48,738 [salt.utils.gitfs :919 ][DEBUG   ][176464] Set update lock for gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 15:57:48,738 [salt.utils.gitfs :817 ][DEBUG   ][176464] Fetching gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 15:57:49,099 [salt.utils.gitfs :1917][DEBUG   ][176464] gitfs remote 'https://github.com/DataDog/datadog-formula.git' is up-to-date
2023-07-07 15:57:49,101 [salt.utils.gitfs :697 ][DEBUG   ][176464] Removed update lock for gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 15:58:19,103 [salt.master      :450 ][DEBUG   ][176464] Updating gitfs fileserver cache for the following targets: [('https://github.com/DataDog/datadog-formula.git', None)]
2023-07-07 15:58:19,103 [salt.utils.gitfs :2885][DEBUG   ][176464] Re-using gitfs object for process 176464
2023-07-07 15:58:19,104 [salt.utils.gitfs :919 ][DEBUG   ][176464] Set update lock for gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 15:58:19,105 [salt.utils.gitfs :817 ][DEBUG   ][176464] Fetching gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 15:58:19,451 [salt.utils.gitfs :1917][DEBUG   ][176464] gitfs remote 'https://github.com/DataDog/datadog-formula.git' is up-to-date
2023-07-07 15:58:19,453 [salt.utils.gitfs :697 ][DEBUG   ][176464] Removed update lock for gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 15:58:49,455 [salt.master      :450 ][DEBUG   ][176464] Updating gitfs fileserver cache for the following targets: [('https://github.com/DataDog/datadog-formula.git', None)]
2023-07-07 15:58:49,456 [salt.utils.gitfs :2885][DEBUG   ][176464] Re-using gitfs object for process 176464
2023-07-07 15:58:49,458 [salt.utils.gitfs :919 ][DEBUG   ][176464] Set update lock for gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 15:58:49,458 [salt.utils.gitfs :817 ][DEBUG   ][176464] Fetching gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 15:58:49,802 [salt.utils.gitfs :1917][DEBUG   ][176464] gitfs remote 'https://github.com/DataDog/datadog-formula.git' is up-to-date
2023-07-07 15:58:49,805 [salt.utils.gitfs :697 ][DEBUG   ][176464] Removed update lock for gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 15:59:19,807 [salt.master      :450 ][DEBUG   ][176464] Updating gitfs fileserver cache for the following targets: [('https://github.com/DataDog/datadog-formula.git', None)]
2023-07-07 15:59:19,807 [salt.utils.gitfs :2885][DEBUG   ][176464] Re-using gitfs object for process 176464
2023-07-07 15:59:19,808 [salt.utils.gitfs :919 ][DEBUG   ][176464] Set update lock for gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 15:59:19,809 [salt.utils.gitfs :817 ][DEBUG   ][176464] Fetching gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 15:59:20,160 [salt.utils.gitfs :1917][DEBUG   ][176464] gitfs remote 'https://github.com/DataDog/datadog-formula.git' is up-to-date
2023-07-07 15:59:20,162 [salt.utils.gitfs :697 ][DEBUG   ][176464] Removed update lock for gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 15:59:50,164 [salt.master      :450 ][DEBUG   ][176464] Updating gitfs fileserver cache for the following targets: [('https://github.com/DataDog/datadog-formula.git', None)]
2023-07-07 15:59:50,165 [salt.utils.gitfs :2885][DEBUG   ][176464] Re-using gitfs object for process 176464
2023-07-07 15:59:50,166 [salt.utils.gitfs :919 ][DEBUG   ][176464] Set update lock for gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 15:59:50,167 [salt.utils.gitfs :817 ][DEBUG   ][176464] Fetching gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 15:59:50,502 [salt.utils.gitfs :1917][DEBUG   ][176464] gitfs remote 'https://github.com/DataDog/datadog-formula.git' is up-to-date
2023-07-07 15:59:50,503 [salt.utils.gitfs :697 ][DEBUG   ][176464] Removed update lock for gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 16:00:20,505 [salt.master      :450 ][DEBUG   ][176464] Updating gitfs fileserver cache for the following targets: [('https://github.com/DataDog/datadog-formula.git', None)]
2023-07-07 16:00:20,505 [salt.utils.gitfs :2885][DEBUG   ][176464] Re-using gitfs object for process 176464
2023-07-07 16:00:20,506 [salt.utils.gitfs :919 ][DEBUG   ][176464] Set update lock for gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 16:00:20,506 [salt.utils.gitfs :817 ][DEBUG   ][176464] Fetching gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 16:00:20,859 [salt.utils.gitfs :1917][DEBUG   ][176464] gitfs remote 'https://github.com/DataDog/datadog-formula.git' is up-to-date
2023-07-07 16:00:20,861 [salt.utils.gitfs :697 ][DEBUG   ][176464] Removed update lock for gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 16:00:50,862 [salt.master      :450 ][DEBUG   ][176464] Updating gitfs fileserver cache for the following targets: [('https://github.com/DataDog/datadog-formula.git', None)]
2023-07-07 16:00:50,863 [salt.utils.gitfs :2885][DEBUG   ][176464] Re-using gitfs object for process 176464
2023-07-07 16:00:50,864 [salt.utils.gitfs :919 ][DEBUG   ][176464] Set update lock for gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 16:00:50,865 [salt.utils.gitfs :817 ][DEBUG   ][176464] Fetching gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 16:00:51,212 [salt.utils.gitfs :1917][DEBUG   ][176464] gitfs remote 'https://github.com/DataDog/datadog-formula.git' is up-to-date
2023-07-07 16:00:51,214 [salt.utils.gitfs :697 ][DEBUG   ][176464] Removed update lock for gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 16:01:21,216 [salt.master      :450 ][DEBUG   ][176464] Updating gitfs fileserver cache for the following targets: [('https://github.com/DataDog/datadog-formula.git', None)]
2023-07-07 16:01:21,217 [salt.utils.gitfs :2885][DEBUG   ][176464] Re-using gitfs object for process 176464
2023-07-07 16:01:21,218 [salt.utils.gitfs :919 ][DEBUG   ][176464] Set update lock for gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 16:01:21,218 [salt.utils.gitfs :817 ][DEBUG   ][176464] Fetching gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 16:01:21,593 [salt.utils.gitfs :1917][DEBUG   ][176464] gitfs remote 'https://github.com/DataDog/datadog-formula.git' is up-to-date
2023-07-07 16:01:21,594 [salt.utils.gitfs :697 ][DEBUG   ][176464] Removed update lock for gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 16:01:51,595 [salt.master      :450 ][DEBUG   ][176464] Updating gitfs fileserver cache for the following targets: [('https://github.com/DataDog/datadog-formula.git', None)]
2023-07-07 16:01:51,595 [salt.utils.gitfs :2885][DEBUG   ][176464] Re-using gitfs object for process 176464
2023-07-07 16:01:51,596 [salt.utils.gitfs :919 ][DEBUG   ][176464] Set update lock for gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 16:01:51,596 [salt.utils.gitfs :817 ][DEBUG   ][176464] Fetching gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 16:01:51,935 [salt.utils.gitfs :1917][DEBUG   ][176464] gitfs remote 'https://github.com/DataDog/datadog-formula.git' is up-to-date
2023-07-07 16:01:51,938 [salt.utils.gitfs :697 ][DEBUG   ][176464] Removed update lock for gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 16:03:28,736 [salt.utils.gitfs :2775][DEBUG   ][171362] pygit2 gitfs_provider enabled
2023-07-07 16:03:28,736 [salt.utils.gitfs :2879][DEBUG   ][171362] Created gitfs object with uninitialized remotes
2023-07-07 16:03:28,736 [salt.loader.lazy :963 ][DEBUG   ][171362] The functions from module 'gitfs' are being loaded by dir() on the loaded module
2023-07-07 16:03:28,737 [salt.utils.lazy  :99  ][DEBUG   ][171362] LazyLoaded gitfs.envs
2023-07-07 16:03:28,748 [salt.utils.gitfs :2885][DEBUG   ][171362] Re-using gitfs object for process 171362
2023-07-07 16:03:28,780 [salt.master      :450 ][DEBUG   ][178713] Updating gitfs fileserver cache for the following targets: [('https://github.com/saltstack/salt-bootstrap.git', None)]
2023-07-07 16:03:28,783 [salt.master      :450 ][DEBUG   ][178713] Updating gitfs fileserver cache for the following targets: [('https://github.com/DataDog/datadog-formula.git', None)]
2023-07-07 16:03:28,784 [salt.utils.gitfs :2885][DEBUG   ][178713] Re-using gitfs object for process 178713
2023-07-07 16:03:28,784 [salt.utils.gitfs :2885][DEBUG   ][178713] Re-using gitfs object for process 178713
2023-07-07 16:03:28,787 [salt.utils.gitfs :919 ][DEBUG   ][178713] Set update lock for gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 16:03:28,788 [salt.utils.gitfs :817 ][DEBUG   ][178713] Fetching gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 16:03:28,788 [salt.utils.gitfs :919 ][DEBUG   ][178713] Set update lock for gitfs remote 'https://github.com/saltstack/salt-bootstrap.git'
2023-07-07 16:03:28,789 [salt.utils.gitfs :817 ][DEBUG   ][178713] Fetching gitfs remote 'https://github.com/saltstack/salt-bootstrap.git'
2023-07-07 16:03:29,137 [salt.utils.gitfs :1917][DEBUG   ][178713] gitfs remote 'https://github.com/DataDog/datadog-formula.git' is up-to-date
2023-07-07 16:03:29,140 [salt.utils.gitfs :697 ][DEBUG   ][178713] Removed update lock for gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 16:03:29,272 [salt.utils.gitfs :1917][DEBUG   ][178713] gitfs remote 'https://github.com/saltstack/salt-bootstrap.git' is up-to-date
2023-07-07 16:03:29,276 [salt.utils.gitfs :697 ][DEBUG   ][178713] Removed update lock for gitfs remote 'https://github.com/saltstack/salt-bootstrap.git'
2023-07-07 16:03:59,142 [salt.master      :450 ][DEBUG   ][178713] Updating gitfs fileserver cache for the following targets: [('https://github.com/DataDog/datadog-formula.git', None)]
2023-07-07 16:03:59,143 [salt.utils.gitfs :2885][DEBUG   ][178713] Re-using gitfs object for process 178713
2023-07-07 16:03:59,145 [salt.utils.gitfs :919 ][DEBUG   ][178713] Set update lock for gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 16:03:59,146 [salt.utils.gitfs :817 ][DEBUG   ][178713] Fetching gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 16:03:59,504 [salt.utils.gitfs :1917][DEBUG   ][178713] gitfs remote 'https://github.com/DataDog/datadog-formula.git' is up-to-date
2023-07-07 16:03:59,506 [salt.utils.gitfs :697 ][DEBUG   ][178713] Removed update lock for gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 16:04:29,508 [salt.master      :450 ][DEBUG   ][178713] Updating gitfs fileserver cache for the following targets: [('https://github.com/DataDog/datadog-formula.git', None)]
2023-07-07 16:04:29,509 [salt.utils.gitfs :2885][DEBUG   ][178713] Re-using gitfs object for process 178713
2023-07-07 16:04:29,510 [salt.utils.gitfs :919 ][DEBUG   ][178713] Set update lock for gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 16:04:29,510 [salt.utils.gitfs :817 ][DEBUG   ][178713] Fetching gitfs remote 'https://github.com/DataDog/datadog-formula.git'
2023-07-07 16:04:29,874 [salt.utils.gitfs :1917][DEBUG   ][178713] gitfs remote 'https://github.com/DataDog/datadog-formula.git' is up-to-date
2023-07-07 16:04:29,877 [salt.utils.gitfs :697 ][DEBUG   ][178713] Removed update lock for gitfs remote 'https://github.com/DataDog/datadog-formula.git'

Expected behavior
I would expect all GitFS remotes to sync on their specified interval without interruption, regardless of the update_interval of other remotes. I find it strange that syncing resumes after the repo with the longest update interval syncs.

Versions Report

salt --versions-report (Provided by running salt --versions-report. Please also mention any differences in master/minion versions.) 
Salt Version:
          Salt: 3005.1

Dependency Versions:
          cffi: 1.15.1
      cherrypy: Not Installed
      dateutil: 2.6.1
     docker-py: Not Installed
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 2.10.1
       libgit2: 1.1.0
      M2Crypto: 0.35.2
          Mako: 1.0.6
       msgpack: 0.6.2
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     pycparser: 2.21
      pycrypto: Not Installed
  pycryptodome: Not Installed
        pygit2: 1.6.1
        Python: 3.6.8 (default, Jan 23 2023, 22:31:05)
  python-gnupg: Not Installed
        PyYAML: 3.12
         PyZMQ: 20.0.0
         smmap: Not Installed
       timelib: Not Installed
       Tornado: 4.5.3
           ZMQ: 4.3.4

System Versions:
          dist: rhel 8.8 Ootpa
        locale: UTF-8
       machine: x86_64
       release: 4.18.0-477.13.1.el8_8.x86_64
        system: Linux
       version: Red Hat Enterprise Linux 8.8 Ootpa

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.