saltstack / saltstack/salt

[BUG] 3006.x / 3007.x Windows minion throws duplicate custom event in multimaster config

Open
#66,933 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug needs-triage
Dominant language
Python
Stars
15.7k
Forks
5.6k
Avg merge
2d 44m
Merged PRs (30d)
80

Description

Description
Windows minion (3006.x and 3007.1) configured in multimaster config (two masters) throws duplicate custom event when using salt-call event.send or the event.send state module. causing duplicate reactor executions on the same master

salt-call event.send hello/world


salt/auth	{
    "_stamp": "2024-09-27T22:35:27.398730",
    "act": "accept",
    "id": "minion-win-1",
    "pub": "-----BEGIN PUBLIC KEY-----\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAt5l76Z2TNrTeSZJOK7ga\nTUrvQozspW3shL7TYSfGGFJzsY9GD4GBYF9UO68I5/jPYHWh+8se+8QF8mh6y4TQ\nbQU4GN4oSKCAyT8CdwjZnQSpqtgDHenEgI8a4oX1yYPatt/EtGj3MKvMSn5VMVVn\nPnsPzWWS8r25wkwHVDadP2oZrO51fM6niEKbCoXOtgwIp4dzOJMpaZU4Jeu2f+BJ\n3tr5E8W7Q/zJoBEacjlKfrrKwzfS2Ygu9+lefAc4VX5ug9wFgst0W/bmRHa9Ma0L\nNCvzUY0+WFI4qVudsijxOVpvY7nzJTunyCgmfXbJB7968nKHds6g4+xvR7qJ2JPZ\n/wIDAQAB\n-----END PUBLIC KEY-----",
    "result": true
}
minion/refresh/minion-win-1	{
    "Minion data cache refresh": "minion-win-1",
    "_stamp": "2024-09-27T22:35:27.796261"
}
hello/world	{
    "_stamp": "2024-09-27T22:35:27.964432",
    "cmd": "_minion_event",
    "data": {
        "__pub_fun": "event.send",
        "__pub_jid": "20240927223527882312",
        "__pub_pid": 3700,
        "__pub_tgt": "salt-call"
    },
    "id": "minion-win-1",
    "tag": "hello/world"
}
hello/world	{
    "_stamp": "2024-09-27T22:35:27.989619",
    "cmd": "_minion_event",
    "data": {
        "__pub_fun": "event.send",
        "__pub_jid": "20240927223527882312",
        "__pub_pid": 3700,
        "__pub_tgt": "salt-call"
    },
    "id": "minion-win-1",
    "tag": "hello/world"
}
salt/job/20240927223528011683/ret/minion-win-1	{
    "_stamp": "2024-09-27T22:35:28.012717",
    "arg": [
        "hello/world"
    ],
    "cmd": "_return",
    "fun": "event.send",
    "fun_args": [
        "hello/world"
    ],
    "id": "minion-win-1",
    "jid": "20240927223528011683",
    "retcode": 0,
    "return": true,
    "tgt": "minion-win-1",
    "tgt_type": "glob"
}

A few more findings, the issue seems caused by the combination of salt-call and event.send module.

If making the call from the master, using instead salt <minion> event.send or executing a state with the event.send module, the event is generated only once on each master, but on both masters, which I also think it's wrong, the event should be seen only in the master making the call like in other modules.
For instance, using another module like test.echo or test.ping, the return event is seen only in the calling master event bus, not in the other one.

using salt-call with --master option sends the event only to the desired master
salt-call event.send --master=172.21.0.11 some/event

Setup
I've seen these issues in 3006 and 3007, after 3006.8 a similar issue with the /start event seemed fixed.
I'm testing this now with 3007.1 and Windows minion.

Expected behavior
Only the first active master should get only one event

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the duplicate event with a Windows minion in a two-master multimaster configuration, using salt-call event.send and the event.send state module. Compare this with salt-call event.send --master=... and master-side calls, then trace the event.send entry point and event bus handling. Done means one event reaches only the first active master.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.