Graylog2 / Graylog2/graylog2-server

pipeline rules get executed even if you have an and and it shouldn't match

Open
#12,393 2 comments 0 reactions 0 assignees View on GitHub
bug verified
Dominant language
Java
Stars
8.1k
Forks
1.1k
Avg merge
1d 20h
Merged PRs (30d)
217

Description

## Expected Behavior

I expect the rule to only get triggered if all the parameters are true (not one or the other).

## Current Behavior

The rule gets triggered for nginx and mediawiki message. It should only be triggered for the mediawiki one.

But it's getting executed on messages it shouldn't be (for example nginx).

## Steps to Reproduce (for bugs)

1. Create a rule with:

```
rule "mediawiki parse json"
when
has_field("message") and
starts_with(to_string($message.message), "{") and
ends_with(to_string($message.message), "}") and
contains(to_string($message.message), "application_name", true) and
contains(to_string($message.message), "rsyslog", true)
then
let json = parse_json(to_string($message.message));
let map = to_map(json);
//debug(map);
//set_field("message", map);
set_fields(map);

set_field("application_name", to_string("mediawiki"));
//set_field("source", to_string($message.host));

let prop = select_jsonpath(json, {timestamp: "$.@timestamp"});
let new_date = flex_parse_date(to_string(prop.timestamp));
set_field("timestamp", new_date);
end
```

2. Send a message with the contents:

```
{"timestamp":"2022-04-05T14:33:11+00:00", "message":" { \"timestamp\": \"1649169191.528\", \"remote_addr\": \"xxxx\", \"remote_user\": \"\", \"time_local\": \"05\/Apr\/2022:14:33:11 +0000\", \"request_method\": \"GET\", \"request_uri\": \"xxxx\", \"status\": \"304\", \"body_bytes_sent\": \"0\",\"http_x_forwarded_for\": \"xxxx, 127.0.0.1\",\"http_referrer\": \"\", \"http_user_agent\": \"xxx\", \"request_time\": \"0.297\", \"ssl_protocol\": \"TLSv1.3\", \"ssl_cipher\": \"TLS_AES_256_GCM_SHA384\", \"nginx_access_log\": true }", "host":"test101", "logsource":"test101", "severity":"info", "facility":"local7", "application_name":"nginx", "rsyslog": "true"}
```

and

```
{"timestamp":"2022-04-05T14:33:12.264358+00:00", "message":"33:12 mediawiki: {\"@timestamp\":\"2022-04-05T14:33:12.240509+00:00\",\"@version\":1,\"host\":\"test101\",\"message\":\"\",\"type\":\"mediawiki\",\"channel\":\"api-request\",\"level\":\"INFO\",\"monolog_level\":200,\"shard\":\"c4\",\"url\":\"\/w\/api.php\",\"ip\":\"xxxx\",\"http_method\":\"POST\",\"server\":\"xxx\",\"referrer\":\"xxx\",\"wiki\":\"betawiki\",\"mwversion\":\"1.37.2\",\"reqId\":\"xxxx\",\"$schema\":\"\/mediawiki\/api\/request\/1.0.0\",\"meta\":{\"request_id\":\"c098eb2b22dc010cbbd4b071\",\"id\":\"2823cc93-e94c-47a6-8558-55559625110e\",\"dt\":\"2022-04-05T14:33:12Z\",\"domain\":\"beta.betaheze.org\",\"stream\":\"mediawiki.api-request\"},\"http\":{\"method\":\"POST\",\"client_ip\":\"xxx\",\"request_headers\":{\"accept-language\":\"en-GB,en-US;q=0.9,en;q=0.8\",\"referer\":\"xxx\",\"user-agent\":\"xxx\"}},\"performer\":{\"user_text\":\"xxx\"},\"database\":\"betawiki\",\"backend_time_ms\":45,\"params\":{\"action\":\"query\",\"format\":\"json\",\"pageids\":\"1\",\"prop\":\"pagerating\",\"prcontest\":\"\"},\"private\":true}", "host":"localhost", "logsource":"localhost", "severity":"info", "facility":"user", "application_name":"2022-04-05T14", "rsyslog": "true"}
```

(i have a extractor rule that uses regex so it gets to the json correctly).

## Context

I'm migrating to rsyslog and seems the messages are done differently compared to syslog-ng.

In rsyslog it seems to encode the json message so it has \/ which affects our extractor (it doesn't work) so i figured out i can have graylog unescape it by parsing it and writing it out again.
Although there seems to be some issues as you can see with the MediaWiki message (it seems to be doing parts of the time as the application name so within the rule i want the application name set and not apply to the nginx message).

## Your Environment

* Graylog Version: 4.2.7+879e651
* Java Version: "11.0.14" 2022-01-18
* Elasticsearch Version: 7.16.1
* MongoDB Version: 4.4.10
* Operating System: debian 11
* Browser version: Chrome 100

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.