Graylog2 / Graylog2/graylog2-server

Ability to identify wrong timestamps

Open
#4,067 1 comment 0 reactions 0 assignees View on GitHub
feature triaged
Dominant language
Java
Stars
8.1k
Forks
1.1k
Avg merge
1d 20h
Merged PRs (30d)
217

Description

I have a slight problem with timestamps in messages being way out of line.
The reason is most probably because of a bad syslog-sender or a malformed extractor.
To identify where these messages come from I would like to see a simple way to search a specific index for messages that are timestamped way outside the normal timeframe.

## Expected Behavior
Able to search a specific index for anomalies

## Current Behavior
Currently graylog searches all indexes that may contain data from the specified timeperiod.

## Possible Solution
Either a simple query under system/indices that can be used to find the oldest and newest few messages, or a way to tell the normal search window to only search a specified index.

## Steps to Reproduce (for bugs)
1. Index messages with a bad timestamp/have a buggy extractor
2. try to identify them
3.
4.

## Context
We are currently collecting logs from ~10 000 sources, (linux, windows, switches, firewalls, you name it) and as a result, atleast one of my sources are sending data with a malformed timestamp (or I have a buggy extractor).
I am currently rotating indexes every day, but yesterdays index shows up like this:
![image](https://user-images.githubusercontent.com/23400295/29106825-d8fcf51c-7cdf-11e7-851c-1171d96892c3.png)

As a result graylog will have to search quite a few messages for a simple query like "all messages from the last 5 minutes":
![image](https://user-images.githubusercontent.com/23400295/29106847-f683a46e-7cdf-11e7-9fcb-d14292d3949a.png)

## Your Environment

* Graylog Version: 2.3.0
* Elasticsearch Version: 5.5.1
* MongoDB Version: 3.2.12
* Operating System: Ubuntu Linux 14.04 and 16.04
* Browser version:

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.