saltstack / saltstack/salt

syndic duplicate events upstream...

Open
#55,860 7 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug Core severity-low
Dominant language
Python
Stars
15.7k
Forks
5.6k
Avg merge
2d 44m
Merged PRs (30d)
80

Description

Description of Issue

When fireing an event on a minion that is connected via some syndic, the event (and its payload) gets duplicated on each syndic level, by prefix it with syndic/<name>/.

(see how to reproduce below for details)

Is there any reason to duplicate the payload that way?
I doubt it, so which one of those event is supposed to be properly passed along, and which one should be dropped?

Say we'd have 10 levels of syndic (yeah, that would probably be too much... but well...), we would end up with 1k event for a single event (and that applies to alot of stuffs...)

To be honest, we do have a setup with 3 intermediate syndic levels, that already makes eight times each events...

Setup

Having a master, with possibly multiple levels of syndic...

Lets say we have a master with syndic1 connectede to it, syndic2 connected to syndic1, and minion connected to syndic2

master
+- syndic1
    +- syndic2
        +- minion
Steps to Reproduce Issue

If we launch a symple event.fire from the minion:

minion# salt-call event.fire_master '{"data":"my event data"}' 'foobar'

We get the direct item, duplicated on syndic2 level, and both duplicated on syndic1 level:

# salt-run state.event '*foobar'
foobar  {"_stamp": "2020-01-14T15:03:19.459622", "cmd": "_minion_event", "data": {"data": "my event data"}, "id": "minion", "tag": "foobar"} 
syndic/syndic1/foobar {"_stamp": "2020-01-14T15:03:19.459825", "cmd": "_minion_event", "data": {"data": "my event data"}, "id": "minion", "tag": "foobar"} 
syndic/syndic2/foobar   {"_stamp": "2020-01-14T15:03:19.460018", "cmd": "_minion_event", "data": {"data": "my event data"}, "id": "minion", "tag": "foobar"}
syndic/syndic1/syndic/syndic2/foobar  {"_stamp": "2020-01-14T15:03:19.460217", "cmd": "_minion_event", "data": {"data": "my event data"}, "id": "minion", "tag": "foobar"}

In details...

# On minion, to syndic2
# directly generated by the minion, and passed to its master
foobar

# On syndic2 to syndic1
# received by syndic2, and passed along upstream
foobar
# duplicated by syndic2 and also passed upstream
syndic/syndic2/foobar

# On syndic1 passed to master
# received by syndic1, and passed along upstream
foobar
syndic/syndic2/foobar
# duplicated by syndic1 and also passed upstream
syndic/syndic1/foobar
syndic/syndic1/syndic/syndic2/foobar

# Received on master - 4 times the same payload...
foobar
syndic/syndic2/foobar
syndic/syndic1/foobar
syndic/syndic1/syndic/syndic2/foobar
Versions Report

debian salt package: 2019.2.2+ds-1

Salt Version:
           Salt: 2019.2.2
 
Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 2.5.3
      docker-py: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.9.4
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: 0.24.0
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.5.6
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 2.7.13 (default, Sep 26 2018, 18:42:22)
   python-gnupg: Not Installed
         PyYAML: 3.12
          PyZMQ: 16.0.2
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.4.3
            ZMQ: 4.2.1
 
System Versions:
           dist: debian 9.11 
         locale: ANSI_X3.4-1968
        machine: x86_64
        release: 4.4.115-grsec-cerberhost-0.14.1-rap
         system: Linux
        version: debian 9.11 

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the three-level master/syndic/minion setup with salt-call event.fire_master and observe results using salt-run state.event '*foobar'. Trace how the event moves through each syndic level and establish which duplicated tags should remain; done means the intended behavior is documented and verified for nested syndic forwarding.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.