elastic / elastic/elasticsearch

_simulate_index should remove private settings

Open
#136,992 3 comments 0 reactions 0 assignees View on GitHub
:StorageEngine/Templates >enhancement Team:StorageEngine
Dominant language
Java
Stars
77.9k
Forks
26.1k
PR merge metrics
PR metrics pending

Description

Templates cannot be created with private settings as only Elasticsearch itself may add private settings. But `/_index_template/_simulate_index` can currently return private settings. Since templates themselves cannot contains private settings, it does not make sense for the templates returned by `_simulate_index` to contain private settings. Also, since the results of `_simulate_index` are frequently used to create a template or index, a private settings would result in a failure to create the index or template.

The solutions is likely to just remove any private settings created by settings providers, here: https://github.com/elastic/elasticsearch/blob/bce397f40507c262d8b47c885dea02fa46ac0a93/server/src/main/java/org/elasticsearch/action/admin/indices/template/post/TransportSimulateIndexTemplateAction.java#L294

This issue showed up in a failed Kibana test: https://github.com/elastic/kibana/issues/235296 . This tests attempts to create an index using the results of a `_simulate_index` call. This cannot be fixed within Kibana, since it has no way of determining if a given settings is private or not.

Example showing that `_simulate_index` returns private settings:
```

# Create logsdb index with host.name
curl -X PUT "localhost:9200/_index_template/temp1?pretty" -H 'Content-Type: application/json' -d'
{
"index_patterns": ["my-logsdb-datastream-*"],
"data_stream": {},
"template": {
"settings": {
"index.mode": "logsdb",
"number_of_shards": 1
},
"mappings": {
"properties": {
"timestamp": {
"type": "date",
"format": "strict_date_optional_time||epoch_millis"
},
"message": {
"type": "text"
},
"host.name": { "type": "keyword"}
}
}
},
"priority": 200
}'

curl -XPUT localhost:9200/_data_stream/my-logsdb-datastream-5

# sort_on_host_name (which is private) was set
curl localhost:9200/.ds-my-logsdb-datastream-5-2025.10.22-000001/_settings
>>>
...
"logsdb": {
"sort_on_host_name": "true"
},

# Simulate_index also contains private setting
curl -XPOST "localhost:9200/_index_template/_simulate_index/my-logsdb-datastream-5" | jq
>>>
"template": {
"settings": {
"index": {
"mode": "logsdb",
"routing": {
"allocation": {
"include": {
"_tier_preference": "data_hot"
}
}
},
"number_of_shards": "1",
"logsdb": {
"sort_on_host_name": "true"
}
}
},
...

```

Contributor guide

Open the contributing guide

Research direction

Start in server/src/main/java/org/elasticsearch/action/admin/indices/template/post/TransportSimulateIndexTemplateAction.java around line 294, where settings providers contribute to the simulated template. Reproduce the example with _index_template/_simulate_index and verify that the response no longer includes private settings and can be used to create a template or index.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
api, backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.