Missing patterns file causes 100%cpu
- Dominant language
- Java
- Stars
- 14.9k
- Forks
- 3.5k
- Avg merge
- 19h 14m
- Merged PRs (30d)
- 63
Description
Observed that when running logstash on ubuntu, having missing patterns causes logstash to consume 100% cpu indefinitely when restarted. This causes a major problem if the pattern file is not put in place before the config that uses the patterns in it.
Encountered when installing fail2ban configuration.
Example expected patterns file:
```
F2B_DATE %{YEAR}-%{MONTHNUM}-%{MONTHDAY}[ ]%{HOUR}:?%{MINUTE}(?::?%{SECOND})
F2B_ACTION (\w+)\.(?:\w+)(\s+)?\:
F2B_JAIL \[(?\w+\-?\w+?)\]
F2B_LEVEL (?\w+)\s+
```
A corresponding match action in a config file such as below that uses these:
```
input {
file {
type => "fail2ban"
start_position => "beginning"
path => [ "/var/log/fail2ban.log" ]
}
}
filter {
if [type] == "fail2ban" {
grok {
patterns_dir => "/etc/logstash/patterns"
match => [
"message", "%{F2B_DATE:date} %{F2B_ACTION} %{WORD:level} %{F2B_JAIL} %{WORD:action} %{IP:ip}",
"message", "%{F2B_DATE:date} %{F2B_ACTION} %{F2B_LEVEL} %{GREEDYDATA:msg}?"
]
}
geoip {
source => "ip"
}
}
}
```
Resolution: Change logstash not to consume 100% cpu when encountering tokens like F2B_DATE when that pattern has not been defined; an error should be thrown that the input {} configuration is being ignored because it used an unknown pattern.
Contributor guide
Assessment
This issue has not been assessed yet.