Graylog2 / Graylog2/graylog2-server

Graylog 3.2: rsyslog parsing errors

Open
#8,780 2 comments 0 reactions 0 assignees View on GitHub
bug processing triaged
Dominant language
Java
Stars
8.1k
Forks
1.1k
Avg merge
1d 20h
Merged PRs (30d)
217

Description

When parsing rsyslog messages the built-in field extraction (not with extractors or pipeline) provides wrong content

## Expected Behavior
When getting rsyslog messages the fields (timestamp, level, facility, message ...) should be extracted correctly (or NO extraction)

## Current Behavior
Graylog told me of Indexing errors, so I investigated. Graylog (elasticsearch) expected a Number in field "level" but was filled with a string ("error", "warning", ...) which is correct!
Additionally the field "message" (in same message) was not parsed correctly. Parsing stopped after first backslash `("\")` which can be seen in extracted field "msg".

## Possible Solution
1. Parsing field "message" should contain whole message, not stopping at first backslash
2. Field "level" should renamed to "severity" (analog to rsyslog naming convention)

## Steps to Reproduce (for bugs)
In our case it was found per accident. Telegraf was not configured correctly, so it (and docker daemon) threw these messages. No extractor and no pipeline used. Only a static field produced by input.

**1. Content /var/log/messages:**
`Aug 12 09:33:00 infra-dev-test-fixed-2 dockerd: time="2020-08-12T09:33:00.370576643+02:00" level=error msg="Handler for GET /v1.21/services returned error: This node is not a swarm manager. Use \"docker swarm init\" or \"docker swarm join\" to connect this node to swarm and try again."`
`Aug 12 09:33:00 infra-dev-test-fixed-2 telegraf: 2020-08-12T07:33:00Z E! [inputs.docker] Error in plugin: Error response from daemon: This node is not a swarm manager. Use "docker swarm init" or "docker swarm join" to connect this node to swarm and try again.`

**2. Field contents in Graylog (for both messages see above)**
**first message:**
application_name="dockerd"
facility="system daemon"
**level=6**
message=(see messages above)
msg=`Handler for GET /v1.21/services returned error: This node is not a swarm manager. Use \`
time=(ok)
timestamp=(ok)

**second message:**
application_name="telegraf"
facility="system daemon"
**level=6**
message=`2020-08-12T12:46:00Z E! [inputs.docker] Error in plugin: Error response from daemon: This node is not a swarm manager. Use "docker swarm init" or "docker swarm join" to connect this node to swarm and try again.`
(no field msg)
timestamp=(ok)
(no field "time")

## Context

## Your Environment

Single Instance in docker
Vsphere-VM (6.7), 16 GB, 4 CPU

* Graylog Version: Graylog 3.2.6+e255fcc (in docker)
* Java Version:
* Elasticsearch Version: 6.8.3 (in docker)
* MongoDB Version: 4.3.2 (in docker)
* Operating System: CentOS 7.7
* Browser version: Chrome Version 84.0.4147.105, IE 11.0.9600.19749

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.