Graylog2 / Graylog2/graylog2-server
Retention strategy for index set, rotation strategy for indvidual index
- Dominant language
- Java
- Stars
- 8.1k
- Forks
- 1.1k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 217
Description
## What?
It would be hugely beneficial for larger deployments to be able to set a retention strategy at the index set level while still being able to set a rotation strategy at the individual index level.
In short, it would be the ability to say "this index set should hold at least 1 year of data at all times" but at the same time "each individual index shouldn't be more than 30G".
## Why?
* There are often legal requirements about data retention.
* There are technical requirements about the size of each index (specifically shard) for optimal Elasticsearch performance.
It is difficult to make the two types of requirements are difficult to meet with Graylog's current retention policies types: since legal typically trumps technical, time-based retention must be chosen, and then careful monitoring is required to make sure each index is a roughly appropriate size.
Contributor guide
Assessment
This issue has not been assessed yet.