saltstack / saltstack/salt

[BUG] Watchdog generates multiple file modify events

Open
#64,684 11 comments 0 reactions 1 assignee View on GitHub

@twangboy is already working on this.

Since Aug 24, 2023.

bug needs-triage
Dominant language
Python
Stars
15.7k
Forks
5.6k
Avg merge
2d 44m
Merged PRs (30d)
80

Description

Description

If you modify a file that is being watched by watchdog, two beacon events are being generated by the minion. This appears to be because, on windows, multiple events are sent from the windows kernel, through watchdog.

IHAC that configured a watchdog beacon (interval 10) that is always generating two modify events and, as a result, is trigging a reactor twice to do the same work (effectively doubling the load on the salt-minion and the reactor system.

Setup

  1. Install minion on a windows server
  2. Ensure watchdog py module has not been installed before
  3. open console on salt-master and watch event stream ( salt-run state.event pretty=true )
  4. Run the following...
cat <<EOF > /srv/salt/watchdogtest.sls
ensure watchdog installed:
  pip.installed:
    - name: watchdog
    - reload_modules: True
      
watch_dog_beacon:
   beacon.present:
     - name: watchdog
     - save: True
     - enabled: True
     - directories:
        C:\inetpub\wwwroot:
          mask:
            - modify
            - delete
     - interval: 10
     - disable_during_state_run: True
     - require:
       - ensure watchdog installed
  1. salt <windows minion> state.apply watchdogtest
  2. Restart the salt-minion (note: This appears to be another bug - the windows minion can't seem to load the watchdog py module without a restart, causing the beacon.present to fail).

You will see two events go across the bus. And if you have a reactor configured to listen to these events, it will trigger twice.

Please be as specific as possible and give set-up details.

  • on-prem machine
  • VM (Virtualbox, KVM, etc. please specify)
  • VM running on a cloud service, please be explicit and add details
  • container (Kubernetes, Docker, containerd, etc. please specify)
  • or a combination, please be explicit
  • jails if it is FreeBSD
  • classic packaging
  • onedir packaging
  • used bootstrap to install

Steps to Reproduce the behavior

  1. Setup the system (per above)
  2. echo 1 > C:\inetpub\wwwroot\test (no event generated)
  3. wait a few seconds to ensure no bus events are forthcoming....
  4. echo 1 > C:\inetpub\wwwroot\test

As you watch the event bus, you'll see two events are generated

Expected behavior

Should only receive one event, not two. We should be de-duping events from watchdog.

Versions Report

salt --versions-report (Provided by running salt --versions-report. Please also mention any differences in master/minion versions.)
Salt Version:
          Salt: 3005.1

Dependency Versions:
          cffi: 1.14.6
      cherrypy: 18.6.1
      dateutil: 2.8.1
     docker-py: Not Installed
         gitdb: 4.0.7
     gitpython: Not Installed
        Jinja2: 3.1.0
       libgit2: Not Installed
      M2Crypto: Not Installed
          Mako: 1.1.4
       msgpack: 1.0.2
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     pycparser: 2.21
      pycrypto: Not Installed
  pycryptodome: 3.10.1
        pygit2: Not Installed
        Python: 3.8.15 (tags/v3.8.15:44adf8a, Nov  8 2022, 17:20:07) [MSC v.1929 64 bit (AMD64)]
  python-gnupg: 0.4.8
        PyYAML: 5.4.1
         PyZMQ: 22.0.3
         smmap: 4.0.0
       timelib: 0.2.4
       Tornado: 4.5.3
           ZMQ: 4.3.4

System Versions:
          dist:
        locale: cp1252
       machine: AMD64
       release: 2019Server
        system: Windows
       version: 2019Server 10.0.17763 SP0 Multiprocessor Free

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.