Graylog2 / Graylog2/graylog2-server
improved elasticsearch index management (ILM)
- Dominant language
- Java
- Stars
- 8.1k
- Forks
- 1.1k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 217
Description
## What?
Have a more flexible elasticsearch managment and allow hot/warm configurations from within the Graylog UI.
Modern Elasticsearch versions would allow to write ILM configurations ( https://www.elastic.co/guide/en/elasticsearch/reference/current/overview-index-lifecycle-management.html ) like
```
PUT _ilm/policy/graylog_ilm
{
"policy": {
"phases": {
"hot": {
"min_age": "0ms",
"actions": {
"set_priority": {
"priority": 100
}
}
},
"warm": {
"min_age": "8d",
"actions": {
"forcemerge": {
"max_num_segments": 1
},
"set_priority": {
"priority": 50
},
"shrink": {
"number_of_shards": 4
}
}
},
"cold": {
"min_age": "32d",
"actions": {
"freeze": {}
}
}
}
}
}
```
Making use of the given abilities in Elasticsearch would make it easier to implement such a feature and allows user better control of their data storage.
## Why?
It was possible in the past (prior 4.0) to use Elasticsearch ILM ( https://www.elastic.co/guide/en/elasticsearch/reference/current/overview-index-lifecycle-management.html ) to maintain a hot warm cluster. Now the only left solution is to us curator with a configuration like this: https://gist.github.com/jalogisch/12d7e173c555505854a4f9bffbf3155d
This would eliminate the described way we have here: https://www.graylog.org/post/back-to-basics-using-a-hot-warm-elasticsearch-cluster
## Your Environment
* Graylog Version: 4.0
* Elasticsearch Version: 6/7
Contributor guide
Assessment
This issue has not been assessed yet.