[refactor-opportunist] Generate repeated Buildkite beat pipeline scaffolding from shared templates

Score: 8/10 (Impact 3 + Incremental viability 2 + Evidence 2 + reuse bonus 1).

Evidence from recent churn/coupling:
- `git log --since='60 days ago' --format='COMMIT:%H' --name-only -- '*.go' '*.yml' '*.yaml'` found Buildkite beat pipelines repeatedly changing together. Top co-change pairs included `.buildkite/auditbeat/auditbeat-pipeline.yml || .buildkite/metricbeat/pipeline.yml` at 6 commits and many other beat/x-pack pipeline pairs at 5-6 commits.
- Representative co-change commits from `git log --since='60 days ago' --date=short --pretty=format:'%h %ad %s' --name-only -- .buildkite/auditbeat/auditbeat-pipeline.yml .buildkite/metricbeat/pipeline.yml .buildkite/x-pack/pipeline.xpack.filebeat.yml .buildkite/x-pack/pipeline.xpack.metricbeat.yml .buildkite/x-pack/pipeline.xpack.osquerybeat.yml`:
- `3fab654db2 2026-06-22 [main][Automation] Bump VM Image version to 1781496166 (#51141)` touched all five sampled pipelines.
- `9724bfba0c 2026-06-02 [ci][packaging] Update vault-docker-login plugin to v0.8.0 (#51014)` touched all five.
- `79b726c304 2026-05-21 bk: run all beats tests on daily for active branches (#50687)` touched all five.
- `3fe2b55aa7`, `2428610758`, and `8fd03481ad` were additional VM image bumps touching all five.
- The duplicated structures are concrete, not just naming style:
- Shared env/image/plugin scaffolding appears in `.buildkite/auditbeat/auditbeat-pipeline.yml:4-48`, `.buildkite/metricbeat/pipeline.yml:4-48`, `.buildkite/x-pack/pipeline.xpack.filebeat.yml:4-45`, `.buildkite/x-pack/pipeline.xpack.metricbeat.yml:4-48`, and `.buildkite/x-pack/pipeline.xpack.osquerybeat.yml:4-40`.
- The check/update and pre-commit groups repeat the same skeleton with only beat path/context substitutions: `.buildkite/auditbeat/auditbeat-pipeline.yml:50-87`, `.buildkite/metricbeat/pipeline.yml:50-86`, `.buildkite/x-pack/pipeline.xpack.filebeat.yml:47-84`, `.buildkite/x-pack/pipeline.xpack.metricbeat.yml:50-86`, `.buildkite/x-pack/pipeline.xpack.osquerybeat.yml:42-79`.
- Mandatory unit-test steps repeat command/retry/agents/artifacts/test-collector/notify structure: `.buildkite/auditbeat/auditbeat-pipeline.yml:89-170`, `.buildkite/metricbeat/pipeline.yml:88-169`, `.buildkite/x-pack/pipeline.xpack.filebeat.yml:87-168`, `.buildkite/x-pack/pipeline.xpack.metricbeat.yml:88-169`; osquery has the same smaller mandatory scaffold at `.buildkite/x-pack/pipeline.xpack.osquerybeat.yml:81-123`.
- Root trigger definitions duplicate the same if_changed/build-env shape for each beat, for example `.buildkite/pipeline.yml:86-116`, `.buildkite/pipeline.yml:147-172`, `.buildkite/pipeline.yml:174-204`, and `.buildkite/pipeline.yml:206-220`.

Why structural, not cosmetic:
The same platform image versions, Vault plugin versions, retry policies, artifact/test-collector settings, check/update sequencing, and trigger include lists are encoded independently across many pipeline YAML files. Recent churn shows routine infrastructure changes require coordinated edits across files; missing one pipeline can create inconsistent CI coverage or credentials behavior.

Actionable refactor:
Introduce a small checked-in generator or template layer for repeated Buildkite beat pipeline scaffolding, while keeping rendered YAML committed for Buildkite compatibility. Start with data-driven definitions for shared env/plugin blocks plus the check/update, pre-commit, and mandatory Linux unit-test templates. Keep beat-specific sections as explicit overrides.

Representative POC slice:
Render only the shared env/plugin block and the Check/Update group for two low-risk pipelines, e.g. `.buildkite/metricbeat/pipeline.yml` and `.buildkite/x-pack/pipeline.xpack.osquerybeat.yml`, then compare generated output byte-for-byte with the committed YAML for those sections before broadening to mandatory unit-test steps.

Scoped validation command:
After the POC, run the generator check for the two converted pipelines and Buildkite validation if available, e.g. `python3 .buildkite/scripts/render_beat_pipeline.py --check .buildkite/metricbeat/pipeline.yml .buildkite/x-pack/pipeline.xpack.osquerybeat.yml && buildkite-agent pipeline upload --dry-run .buildkite/metricbeat/pipeline.yml && buildkite-agent pipeline upload --dry-run .buildkite/x-pack/pipeline.xpack.osquerybeat.yml`.

---
[What is this?](https://ela.st/github-ai-tools) | [From workflow: Refactor Opportunist](https://github.com/elastic/beats/actions/runs/28382268168)

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

Start with the repeated blocks in .buildkite/metricbeat/pipeline.yml, .buildkite/x-pack/pipeline.xpack.osquerybeat.yml, and the root trigger definitions in .buildkite/pipeline.yml. Implement and run the scoped generator check using the command shown in the issue, then compare rendered output byte-for-byte and run Buildkite dry-run validation if available. Done means the two converted pipelines preserve their committed shared env/plugin and Check/Update YAML while being generated from shared definitions.

Written by the indexing model from the issue text.