NETWAYS / NETWAYS/ansible-collection-elasticstack
[Bug]: Beats setup tasks (filebeat/metricbeat) inconsistent with auditbeat
Nobody has claimed this yet.
- Dominant language
- Jinja
- Stars
- 14
- Forks
- 11
- Avg merge
- 1d 47m
- Merged PRs (30d)
- 6
Description
The three Beats run their setup task (loads index template, ILM, ingest pipelines, dashboards into Elasticsearch) inconsistently, and each has gaps. Auditbeat is the closest to correct and should be the reference for the other two.
Problems
- metricbeat uses bare
metricbeat setup(= index-management + dashboards + ML, but no pipelines), with a marker misleadingly namedpipelines_created. - filebeat is not gated on the Elasticsearch output (fails on the default Logstash output); passes
--pipelines --dashboardsbut no--index-management; modules live inmodules.d/, sosetup --pipelinesfinds none (needs--modules); marker is per-module. - auditbeat uses a run-once-forever marker, so modules/filesets added later never get their pipelines loaded.
Target
- Gate all three on
beats_<beat>_output == "elasticsearch"andrun_once. - Always use explicit flags, never bare
setup, so the task is self-documenting:setup --index-management --pipelines, plus--dashboardsonly when Kibana is configured.--index-managementis safe (setup.template.overwrite: false). - Rename metricbeat's
pipelines_createdmarker to an accurate name (e.g.setup_done). - Replace the run-once-forever marker with re-run on change:
registerthe module/config task and runsetupwhen: <register> is changed. - filebeat: add
--modules(or adopt the inline data-driven design) so module pipelines actually load. - Drop "(experimental)" from the
beats_filebeat_modulesdescription.
Background
With the Elasticsearch output the index template and pipelines auto-load on first run anyway; with the Logstash output nothing auto-loads, so setup is only really needed there — yet the tasks gate on the ES output and skip exactly that case.
The module handling is redesigned data-driven in #510 / #505 / #516 (inline, one list = desired state), which simplifies this further (inline → setup --pipelines sees the modules directly, no --modules). This issue is the fix for the current architecture.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the Beats setup tasks and the module/config tasks referenced in the issue, then review the data-driven redesign in #510, #505, and #516. Compare filebeat, metricbeat, and auditbeat against the explicit setup flags, output and run_once conditions. Done means all three setup flows use the requested gating, markers, rerun behavior, module handling, and description change.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- ansible, elasticsearch
- Domain
- devops, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100