Graylog2 / Graylog2/graylog2-server

Protect Journal Against Corruption

Open
#10,467 0 comments 0 reactions 0 assignees View on GitHub
bug triaged
Dominant language
Java
Stars
8.1k
Forks
1.1k
Avg merge
1d 20h
Merged PRs (30d)
217

Description

Requesting some form of protection for the journal against corruption due to runaway inflation.

When a surge of incoming logs causes the buffer to consume all available space on it's assigned volume, the journal becomes corrupted. This requires deletion of the corrupt journal and the loss of all data contained within.

## Expected Behavior

Journal should never be allowed to exceed it's maximum size setting.
## Current Behavior

Journal will allow journal to exceed max size setting.
## Possible Solution

Shorter intervals for the check, some kind of active measure when it approaches a threshold to prevent corruption from occurring.
## Steps to Reproduce (for bugs)

1.
2.
3.
4.

## Context

The larger the configured journal maximum size, the larger the impact should it become corrupted. In once case, 60GB of collected logs were lost in a single journal deletion.
## Your Environment

* Graylog Version: ALL
* Java Version:
* Elasticsearch Version:
* MongoDB Version:
* Operating System:
* Browser version:

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.