Graylog2 / Graylog2/graylog2-server
Pipeline Processing Error - Change Log Level
- Dominant language
- Java
- Stars
- 8.1k
- Forks
- 1.1k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 217
Description
When a message fails to process in a pipeline the following message is written into the log file:
> WARN [ProcessBufferProcessor] Unable to process message : java.lang.ClassCastException: org.graylog.plugins.pipelineprocessor.ast.expressions.FunctionExpression cannot be cast to org.graylog.plugins.pipelineprocessor.ast.expressions.LogicalExpression
When the above message is written into the log file, the message that it is referencing is never saved within Elasticsearch meaning that the message is lost.
Due to the impact of the above issue, the log level for this message should be Error and not Warning.
Please see [this thread](https://community.graylog.org/t/debugging-message-processing/8652) on the community forum for further reference.
## Expected Behavior
When message processing fails, the message written to the server log file should be an Error level message.
## Current Behavior
When message processing fails, the message written to the server log file is a Warning level message.
## Possible Solution
Change log level for this type of message from Warning to Error.
The ID or Name of the pipeline which this issue occurred in could also be added to the error message.
The pipeline rule editor should also throw an error when a statement like `NOT to_string($message.field)` is present as this rule will not work and will cause errors.
## Steps to Reproduce (for bugs)
Create a pipeline rule with the following statement: `NOT to_string($message.field) == "value"`
When a message hits that rule, the pipeline tries to evaluate `NOT to_string($message.field)` instead of the entire statement.
## Workaround
To get around this specific issue within the Pipeline processor, the above line needs to be wrapped in parentheses, as below:
`NOT (to_string($message.field) == "value")`
## Context
Due to error message not providing any information as to which pipeline or rule this error occurred in, troubleshooting the issue was time consuming as I was pretty much blind as to where the issue was happening. My dev environment isn't massive however, on a system that is more complex and/or larger, this could become a very time consuming issue to investigate.
## Your Environment
* Graylog Version: 2.4.6
* Elasticsearch Version: 5.6.14
* MongoDB Version: 3.6.10
* Operating System: Ubuntu 18.04
* Browser version: Chrome 71.0.3578.98 64-Bit
Contributor guide
Assessment
This issue has not been assessed yet.