Graylog2 / Graylog2/graylog2-server

Pipeline Processing Error - Change Log Level

Open
#5,637 0 comments 0 reactions 0 assignees View on GitHub
bug triaged
Dominant language
Java
Stars
8.1k
Forks
1.1k
Avg merge
1d 20h
Merged PRs (30d)
217

Description

When a message fails to process in a pipeline the following message is written into the log file:

> WARN [ProcessBufferProcessor] Unable to process message : java.lang.ClassCastException: org.graylog.plugins.pipelineprocessor.ast.expressions.FunctionExpression cannot be cast to org.graylog.plugins.pipelineprocessor.ast.expressions.LogicalExpression

When the above message is written into the log file, the message that it is referencing is never saved within Elasticsearch meaning that the message is lost.

Due to the impact of the above issue, the log level for this message should be Error and not Warning.

Please see [this thread](https://community.graylog.org/t/debugging-message-processing/8652) on the community forum for further reference.

## Expected Behavior
When message processing fails, the message written to the server log file should be an Error level message.

## Current Behavior
When message processing fails, the message written to the server log file is a Warning level message.

## Possible Solution
Change log level for this type of message from Warning to Error.
The ID or Name of the pipeline which this issue occurred in could also be added to the error message.
The pipeline rule editor should also throw an error when a statement like `NOT to_string($message.field)` is present as this rule will not work and will cause errors.

## Steps to Reproduce (for bugs)

Create a pipeline rule with the following statement: `NOT to_string($message.field) == "value"`

When a message hits that rule, the pipeline tries to evaluate `NOT to_string($message.field)` instead of the entire statement.

## Workaround

To get around this specific issue within the Pipeline processor, the above line needs to be wrapped in parentheses, as below:

`NOT (to_string($message.field) == "value")`

## Context
Due to error message not providing any information as to which pipeline or rule this error occurred in, troubleshooting the issue was time consuming as I was pretty much blind as to where the issue was happening. My dev environment isn't massive however, on a system that is more complex and/or larger, this could become a very time consuming issue to investigate.

## Your Environment

* Graylog Version: 2.4.6
* Elasticsearch Version: 5.6.14
* MongoDB Version: 3.6.10
* Operating System: Ubuntu 18.04
* Browser version: Chrome 71.0.3578.98 64-Bit

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.