Graylog2 / Graylog2/graylog2-server
Graylog 3.2: rsyslog parsing errors
- Dominant language
- Java
- Stars
- 8.1k
- Forks
- 1.1k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 217
Description
When parsing rsyslog messages the built-in field extraction (not with extractors or pipeline) provides wrong content
## Expected Behavior
When getting rsyslog messages the fields (timestamp, level, facility, message ...) should be extracted correctly (or NO extraction)
## Current Behavior
Graylog told me of Indexing errors, so I investigated. Graylog (elasticsearch) expected a Number in field "level" but was filled with a string ("error", "warning", ...) which is correct!
Additionally the field "message" (in same message) was not parsed correctly. Parsing stopped after first backslash `("\")` which can be seen in extracted field "msg".
## Possible Solution
1. Parsing field "message" should contain whole message, not stopping at first backslash
2. Field "level" should renamed to "severity" (analog to rsyslog naming convention)
## Steps to Reproduce (for bugs)
In our case it was found per accident. Telegraf was not configured correctly, so it (and docker daemon) threw these messages. No extractor and no pipeline used. Only a static field produced by input.
**1. Content /var/log/messages:**
`Aug 12 09:33:00 infra-dev-test-fixed-2 dockerd: time="2020-08-12T09:33:00.370576643+02:00" level=error msg="Handler for GET /v1.21/services returned error: This node is not a swarm manager. Use \"docker swarm init\" or \"docker swarm join\" to connect this node to swarm and try again."`
`Aug 12 09:33:00 infra-dev-test-fixed-2 telegraf: 2020-08-12T07:33:00Z E! [inputs.docker] Error in plugin: Error response from daemon: This node is not a swarm manager. Use "docker swarm init" or "docker swarm join" to connect this node to swarm and try again.`
**2. Field contents in Graylog (for both messages see above)**
**first message:**
application_name="dockerd"
facility="system daemon"
**level=6**
message=(see messages above)
msg=`Handler for GET /v1.21/services returned error: This node is not a swarm manager. Use \`
time=(ok)
timestamp=(ok)
**second message:**
application_name="telegraf"
facility="system daemon"
**level=6**
message=`2020-08-12T12:46:00Z E! [inputs.docker] Error in plugin: Error response from daemon: This node is not a swarm manager. Use "docker swarm init" or "docker swarm join" to connect this node to swarm and try again.`
(no field msg)
timestamp=(ok)
(no field "time")
## Context
## Your Environment
Single Instance in docker
Vsphere-VM (6.7), 16 GB, 4 CPU
* Graylog Version: Graylog 3.2.6+e255fcc (in docker)
* Java Version:
* Elasticsearch Version: 6.8.3 (in docker)
* MongoDB Version: 4.3.2 (in docker)
* Operating System: CentOS 7.7
* Browser version: Chrome Version 84.0.4147.105, IE 11.0.9600.19749
Contributor guide
Assessment
This issue has not been assessed yet.