Incorrect Logstash Configuration for System Logs Documented
- Dominant language
- Java
- Stars
- 14.9k
- Forks
- 3.5k
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 88
Description
Incorrect logstash pipeline configuration for system authorization logs documented at https://www.elastic.co/guide/en/logstash/current/logstash-config-for-filebeat-modules.html#parsing-system
Symptoms:
The following error and failure to index authorization log events:
[2017-06-21T10:02:32,974][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"active-syslog", :_type=>"auth", :_routing=>nil}, 2017-06-06T14:53:38.000Z hostname %{message}], :response=>{"index"=>{"_index"=>"syslog-000001", "_type"=>"auth", "_id"=>"AVzK9jiDxj20MZUklAjK", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [system.auth.user]", "caused_by"=>{"type"=>"illegal_state_exception", "reason"=>"Can't get text on a START_OBJECT at 1:128"}}}}}
Problem:
The documented pipeline configuration creates a [system][auth] object with a [user] field and then attempts to create a [system][auth][user][add] object with various 'useradd' fields. This creates a type collision which causes elasticsearch to fail to index one of the types.
Solution:
Use a different object for 'useradd' events such as [system][auth][useradd]
Contributor guide
Research direction
Start with the linked Logstash documentation section on parsing system logs and inspect the documented pipeline for system authorization and useradd events. Change the documented object path so these events do not collide, then verify that authorization log events can be indexed without the reported mapper parsing exception.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- elasticsearch
- Domain
- documentation
- Issue type
- Documentation
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 45/100