Graylog2 / Graylog2/graylog2-server

Retention strategy for index set, rotation strategy for indvidual index

Open
#9,897 1 comment 2 reactions 0 assignees View on GitHub
feature triaged
Dominant language
Java
Stars
8.1k
Forks
1.1k
Avg merge
1d 20h
Merged PRs (30d)
217

Description

## What?

It would be hugely beneficial for larger deployments to be able to set a retention strategy at the index set level while still being able to set a rotation strategy at the individual index level.

In short, it would be the ability to say "this index set should hold at least 1 year of data at all times" but at the same time "each individual index shouldn't be more than 30G".

## Why?

* There are often legal requirements about data retention.
* There are technical requirements about the size of each index (specifically shard) for optimal Elasticsearch performance.

It is difficult to make the two types of requirements are difficult to meet with Graylog's current retention policies types: since legal typically trumps technical, time-based retention must be chosen, and then careful monitoring is required to make sure each index is a roughly appropriate size.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.