elastic / elastic/logstash

Feature Request: Cancel filtering if event takes longer than a set amount of time

Open
#19,019 0 comments 0 reactions 0 assignees View on GitHub
enhancement status:needs-triage
Dominant language
Java
Stars
14.9k
Forks
3.5k
Avg merge
1d 4h
Merged PRs (30d)
88

Description

For example we are using the RabbitMQ input plugin and sometimes an event seems to get stuck in some filter for some reason. Debugging this is nearly impossible because it's like finding the needle in a huge haystack (in my current case it must be 1 or multiple events in a batch of 30000 and there might others in the 4 million messages that are still in the queue). It's not happening very often, maybe once every few months. I assume that there must be some huge message that is not handled well by some filter plugin other than `grok`, because `grok` has a default timeout of 30s as far as I read in the documentation. While the message is stuck, events jam up in the affected RabbitMQ queue(s) and nothing is consumed anymore (or stuff is consumed very slowly). I have no errors/slowlogs etc. in my Logstash logs indicating that anything is wrong. Also the logs from RabbitMQ look fine. Logstash connects to the affected queues after a restart and i see that it fills its pipeline batch - but then it just is stuck.

So it would be nice to have some general filter config-option that a message is canceled of being filtered further by any filter-plugin if it is taking Logstash too long after a set amount of time in milliseconds. The message then should be either just ingested to some fallback index so you can debug it later or just put out to the logs.

Or is there already some option like that which I missed? I searched in the docs but couldn't find something similar.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.