Graylog2 / Graylog2/graylog2-server

"grok()" pipeline function's "matches" should accurately reflect whether the pattern matches

Open
#24,320 0 comments 0 reactions 0 assignees View on GitHub
bug triaged
Dominant language
Java
Stars
8.1k
Forks
1.1k
Avg merge
1d 20h
Merged PRs (30d)
217

Description

## Expected Behavior

The `matches` property exposed by the `GrokResult` (returned by pipeline function `grok()`) should accurately reflect whether the pattern (i.e. regular expression) matches.

## Current Behavior

The aforementioned grok `matches` is only `true` if there are captures (i.e. [`captures.size() > 0`](https://github.com/Graylog2/graylog2-server/blob/7.0.0/graylog2-server/src/main/java/org/graylog/plugins/pipelineprocessor/functions/strings/GrokMatch.java#L96)).

These Graylog pipeline rule conditions don't result in a rule match despite using valid grok patterns (assume the `message` field exists with a value of `Hello World`):
* `grok(".+", to_string($message.message)).matches == true`
* `grok("%{WORD:UNWANTED}", to_string($message.message)).matches == true`

Here are a couple of simple examples that unintuitively don't result in a pipeline rule match (like above, assume there's a valid field+content-- e.g. `message: 21-11-2025 Hello World`):

```
rule "grok example - UNWANTED in condition for clear non-use"
when
// does not match due to no captures
grok("^\\d{2}-\\d{2}-\\d{4}\\s+%{WORD:UNWANTED}", to_string($message.message)).matches == true
then
// similar grok expression, but with a named capture instead of discarding
set_fields(grok("^\\d{2}-\\d{2}-\\d{4}\\s+%{WORD:some_field}", to_string($message.message)));
end
```

```
rule "grok example - temporarily testing using a plain regex"
when
// does not match due to no captures
grok("^\\d{2}-\\d{2}-\\d{4}\\s+\\w+", to_string($message.message)).matches == true
then
// do stuff
end
```

## Possible Solution

Use `match.getMatch().matches()` (or `match.getMatch().find()`) for the value of `matches` in `GrokResult`, making behavior consistent with `matches` from `RegexMatchResult` (returned by pipeline function `regex()`).

https://github.com/graylog-labs/java-grok/blob/grok-0.1.9-graylog-3/src/main/java/io/krakens/grok/api/Match.java

## Steps to Reproduce (for bugs)

1. Run the pipeline rule simulator using the sample content + rules mentioned above

## Context

Our standard approach to use of the `grok()` pipeline function is to have _almost_ the same `grok()` expression in both the pipeline rule condition and body-- with the only difference being the condition explicitly discards values via the `UNWANTED` identifier. This lets us:
* Clearly indicate non-use of values in the rule condition.
* Ensure the expected content (i.e. captures) will be available for use in the rule body.

We recently created (surprisingly for the first time) a pipeline rule using the aforementioned approach where there weren't _any_ captures in the condition (we typically use at least `TIMESTAMP_ISO8601`), which led to discovering the actual behavior of `grok()`.

## Your Environment

* Graylog Version: `7.0.0`

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.