Graylog2 / Graylog2/graylog2-server
data loss when pipeline rule throws an exception
- Dominant language
- Java
- Stars
- 8.1k
- Forks
- 1.1k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 217
Description
This is the more general issue raised in #5405.
## Expected behavior
Any message arriving in Graylog that is not explicitly discarded by the admin should end up in the database, no matter what.
## Current behavior
If a Java exception is thrown during the execution of a rule in a pipeline, processing of that message cedes immediately and the message is lost. It is not stored in the database.
## Possible Solution
I would argue that data must never be lost. Instead the current pipeline should be aborted, but message processing should continue somehow, at least end up in the database and the `All messages` stream.
Additionally the admin should be notified, both by email (there's a `root_email` attribute in the `server.yml` config file, after all) and by popup error messages (similar to e.g. when an input fails to start). At the moment the exception is reduced to a warning in the log file, and the fact that data loss occurred is not mentioned anywhere.
## Steps to Reproduce (for bugs)
Please see #5405 for detailed instructions how to reproduce the issue. Instead of using the simulator, simply create the pipeline & rule as mentioned there, connect it to the `All messages` stream and create a message containing solely an IPv6 address, e.g. via `logger 2001:db8::1`
## Your Environment
* Graylog Version: 2.4.6-1 (Ubuntu package from Graylog's repo)
* Elasticsearch Version: 5.6.13 (Ubuntu package from Elastico's repo)
* MongoDB Version: 4.0.4 (Ubuntu package from MongoDB.org's repo)
* Operating System: Ubuntu 18.04
Contributor guide
Assessment
This issue has not been assessed yet.