Graylog2 / Graylog2/graylog2-server

Support roll-up/correlation of messages

Open
#679 3 comments 7 reactions 0 assignees View on GitHub
feature processing triaged
Dominant language
Java
Stars
8.1k
Forks
1.1k
Avg merge
1d 20h
Merged PRs (30d)
217

Description

Add support for generating data out of multiple other messages, without requiring the operation to be done at indexing time.

For example, take a source that emits start and end events separately. These messages might be processed by two separate graylog2 servers, but the user wants to know the duration between these two events and graph them.

One possible implementation could look like this:
- Given a set of queries/rules, graylog2 identifies messages as being part of a correlation message set.
- The queries/rules optionally order the messages by some arbitrary collation.
- A set of rules describes how to aggregate all or part of the data contained in those messages.
- From this data a new message is created which should be indexed separately.
- New aggregations or visualizations can then be performed using this new data source.

This feature could work in conjunction with streams or drools rules, preferably in an offline fashion (i.e. not hindering initial messages processing), but rather be done at search time or a dedicated (batch) processing node.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.