elastic / elastic/elasticsearch

x-pack/plugin/apm-data creates templates referencing ILM policies that are not created

Open
#114,591 4 comments 0 reactions 0 assignees View on GitHub
:StorageEngine/Data streams >bug Team:StorageEngine
Dominant language
Java
Stars
77.9k
Forks
26.1k
PR merge metrics
PR metrics pending

Description

### Elasticsearch Version

8.15.1

### Installed Plugins

apm-data

### Java Version

openjdk 21.0.4 2024-07-16 LTS OpenJDK Runtime Environment Temurin-21.0.4+7 (build 21.0.4+7-LTS) OpenJDK 64-Bit Server VM Temurin-21.0.4+7 (build 21.0.4+7-LTS, mixed mode, sharing)

### OS Version

Rocky Linux release 9.4 (Blue Onyx)

### Problem Description

When Elasticsearch is configured with `xpack.apm_data.enabled` to enable the creation of APM indices (to support running APM Server standalone, without needing Fleet), the index templates that are created reference ILM policies that do not exist

For example the component template `traces-apm.rum-fallback@lifecycle` is configured with an ILM policy of `traces-apm.rum_traces-default_policy` which does not exist. The setting does include `_prefer_ilm: false` however it seems incorrect behaviour to have references in these templates to policies that do not exist.

I've assumed this is a bug, but it might just be it's not clear the expectation of a user on how to manage ILM policies for these APM data streams

### Steps to Reproduce

- Spin up a barebones Elastic instance/cluster with `xpack.apm_data.enabled: true` in the configuration
- Observe created index & component templates without matching ILM policies

### Logs (if relevant)

_No response_

Contributor guide

Open the contributing guide

Research direction

Start in x-pack/plugin/apm-data, inspect how enabling xpack.apm_data.enabled creates the APM index and component templates, especially traces-apm.rum-fallback@lifecycle. Reproduce with a bare Elasticsearch cluster using the reported setting and check the referenced ILM policies. Done means the template-policy relationship is corrected or the expected ILM management behavior is clearly established and covered by tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.