Graylog2 / Graylog2/graylog2-server

Allow to choose date field in event definition

Open
#15,302 3 comments 0 reactions 0 assignees View on GitHub
feature triaged
Dominant language
Java
Stars
8.1k
Forks
1.1k
Avg merge
1d 20h
Merged PRs (30d)
217

Description

## What?
Searches for events are based on the `timestamp` field.
This field has the timestamp from the time the message arrived in graylog.
There is a second timestamp field named `event_created`.
`event_created` is the timestamp from the original log message.

Add the option for event definitions to choose the date field for the search.

## Why?

At the moment you can only use `event_created` for widgets and searches but not as source timestamp for event definitions.

When you have a bigger cluster production setup or e.g. a kafka message broker cluster in front of graylog it often happens that messages shown up in the wrong order when using the `timestamp` field, because one of the messages arrived a bit later then the other in graylog.

Example
![image](https://user-images.githubusercontent.com/33449202/233553382-baa724a4-2b55-4fed-9614-6cd01096d24d.png)

If you try to do a event correlation with the events and try to solve the problem to find systems where the journal is stopped but not restarted in a specific time frame you will never get a match for the correlation because for graylog the log message with "Journal started" is the first message and "Journal stopped" the second message.

The real timestamp, `event_created`, shows the real world scenario in the correct order.

Even when messages arrived late because of network outages or something else a correlation based on the timestamp when the message arrived in graylog is useless.

I can understand why the `timestamp` field should not be overwritten, but without the ability to correlate `events based` on the event_created field, correlations are often useless in my opinion.

## Your Environment

* Graylog Version: 5.0.6
* Opensearch Version: 2.3
* MongoDB Version: 6.0.5
* Operating System: SLES 15
* Browser version: Any browser

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.