[BUG] Various engines don't forward master events if master id is set
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 15.7k
- Forks
- 5.6k
- Avg merge
- 2d 44m
- Merged PRs (30d)
- 80
Description
Description
When using various salt engines to forward events, master events are not forwarded if the master has an explicitly set id. I saw this with the fluent engine, but suspect it will also be the case for logentries, http_logstash, and logstash_engine, based on:
salt/engines/logentries.py
183: if opts.get("id").endswith("_master"):
salt/engines/http_logstash.py
99: if __opts__.get("id").endswith("_master"):
salt/engines/fluent.py
80: if __opts__.get("id").endswith("_master"):
salt/engines/logstash_engine.py
51: if opts.get("id").endswith("_master"):
Setup
Install dependency:
pip3 install fluent-logger
Master configuration:
id: foo
log_level_logfile: debug
engines:
- fluent:
host: localhost
port: 24224
app: engine
- test: {}
Patch the fluentd engine so you can see if it sends events:
diff --git a/salt/engines/fluent.py b/salt/engines/fluent.py
index 96ba663a52..1a9436379b 100644
--- a/salt/engines/fluent.py
+++ b/salt/engines/fluent.py
@@ -89,9 +89,14 @@ def start(host="localhost", port=24224, app="engine"):
sock_dir=__opts__["sock_dir"],
listen=True,
)
- log.info("Fluent engine started")
+ log.info("Fluent engine started: sending events to {}:{}".format(host, port))
while True:
salt_event = event_bus.get_event_block()
if salt_event:
+ log.debug(
+ "fluentd engine: Sending tag {}".format(
+ salt_event.get("tag", "UNKNOWN")
+ )
+ )
event.Event(app, salt_event)
Steps to Reproduce the behavior
Start the master, look for engine in the logs. The test engine will show events, but there will no fluentd engine: Sending tag logs at the same time.
Expected behavior
Events should be forwarded for explicitly-named masters.
Versions Report
salt --versions-report
(Provided by running salt --versions-report. Please also mention any differences in master/minion versions.)Salt Version:
Salt: 3001.1
Dependency Versions:
cffi: Not Installed
cherrypy: Not Installed
dateutil: Not Installed
docker-py: Not Installed
gitdb: Not Installed
gitpython: Not Installed
Jinja2: 2.11.1
libgit2: Not Installed
M2Crypto: 0.35.2
Mako: Not Installed
msgpack-pure: Not Installed
msgpack-python: 0.6.2
mysql-python: Not Installed
pycparser: Not Installed
pycrypto: Not Installed
pycryptodome: Not Installed
pygit2: Not Installed
Python: 3.6.8 (default, Apr 2 2020, 13:34:55)
python-gnupg: Not Installed
PyYAML: 3.13
PyZMQ: 17.0.0
smmap: Not Installed
timelib: Not Installed
Tornado: 4.5.3
ZMQ: 4.1.4
System Versions:
dist: centos 7 Core
locale: UTF-8
machine: x86_64
release: 5.4.0-47-generic
system: Linux
version: CentOS Linux 7 Core
Additional context
The fix looks to be changing:
if __opts__.get("id").endswith("_master"):
to:
if __opts__.get("__role") == "master":
ZD-5670.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the role checks in salt/engines/fluent.py, salt/engines/logentries.py, salt/engines/http_logstash.py, and salt/engines/logstash_engine.py, then reproduce the fluent setup with an explicitly configured master id. Verify that master events are forwarded by the affected engines when the id is not suffixed with _master; the issue's expected behavior provides the completion check.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- devops, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100