elastic / elastic/logstash

logstash stops processing when an anomaly in event stream is found

Open
#3,269 0 comments 0 reactions 0 assignees View on GitHub
pipeline stalls
Dominant language
Java
Stars
14.9k
Forks
3.5k
Avg merge
19h 14m
Merged PRs (30d)
63

Description

Dont know if this question/bug is

I am ingesting the AWS ELB access logs and at times [due to how ELB logs]; there are entries like this:

```
2015-05-15T21:51:49.335698Z elb-production 31.3.226.2:38778 10.0.55.56:80 0.000041 0.00055 0.000071 301 301 0 184 "GET http://55.48.116.183^M^M
Accept:80/user.php?action=users-Login HTTP/1.1"
```

The above in principle must be a single line and without the carriage [^M^M], but that is how ELB is reporting it and there is nothing we can do about it I guess. But the moment this is encountered, logstash stops processing with an exception.

So my question is
- Is there a grok which can catch this type of anomaly in a otherwise large stream of data.
- If not how can we skip such events and get to the next event. Ideally it would be nice, if logstash would go on to next event.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.