NETWAYS / NETWAYS/ansible-collection-elasticstack

[Feature]: Data-driven generic template for Filebeat modules (replace beats_filebeat_modules)

Open
#510 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

breaking-change component:filebeat feature
Dominant language
Jinja
Stars
14
Forks
11
Avg merge
1d 47m
Merged PRs (30d)
6

Description

Today beats_filebeat_modules is a flat list of module names; the role only runs filebeat modules enable, so a module's config (paths, vars, filesets) can't be customized — only the system module is templated.

Replace the flat beats_filebeat_modules name-list with a data-driven variable (a list of modules with their config), rendered inline into filebeat.yml (filebeat.modules:) by a shared template/macro. This one list is the declarative desired state: modules in the list are enabled and configured; modules removed on a later run disappear — no separate disable list, no enabled: false needed.

Render inline, not into modules.d/*.yml: inline is re-rendered wholesale each run → removal is automatic; per-file modules.d/ rendering would orphan files (module stays active) and need cleanup, and modules.d settings take precedence over the main config. Drop the imperative filebeat modules enable; leave modules.d/ untouched.

Re-adapt the setup task here too (supersedes #508's current-architecture fix): run filebeat setup when this template changes (register/is changed, run_once) so pipelines for newly added filesets load; inline means setup --pipelines sees the modules directly, so the --modules workaround from #508 is no longer needed.

Docs: the module structure is documented once in a shared roles/beats/docs/beats-modules.md (common model + a Filebeat section with examples), linked from the beats_filebeat_modules description; the README table shows only a short default + the pointer.

Breaking: beats_filebeat_modules changes from a name list to a data structure. Shares the macro + docs page with #505 and the Auditbeat modules issue. Related: #393 (filebeat inputs).


Depends on: #393 (shared plugin-as-data macro).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with dependency #393 and the existing Beats role templates and setup task; the issue specifically names inline filebeat.yml rendering and roles/beats/docs/beats-modules.md. Trace how beats_filebeat_modules is currently described and consumed, then verify that the shared data model, inline rendering, setup rerun behavior, removal semantics, and documentation examples all match the requested declarative state without changing modules.d/.

Written by the indexing model from the issue text.

Assessment

Tech stack
ansible
Domain
devops, observability-sre
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.