Graylog2 / Graylog2/graylog2-server

Graylog GROK patterns buffer overflow

Open
#5,155 3 comments 0 reactions 0 assignees View on GitHub
to-verify
Dominant language
Java
Stars
8.1k
Forks
1.1k
Avg merge
1d 20h
Merged PRs (30d)
217

Description

Graylog GROK patterns buffer overflow

## Expected Behavior
We have bad messages(corrupted) from nginx 1.14 error_log size 2048 byte with GROK pattern
`nginx:\s(?%{YEAR}[./-]%{MONTHNUM}[./-]%{MONTHDAY}[- ]%{TIME}) [%{LOGLEVEL:severity}] %{POSINT:pid222}#%{NUMBER}: (?(.|\r|\n)*)(?:, client: %{IPORHOST:remote_addr})(?:, server: %{IPORHOSTORUNDERSCORE:server})(?:, request: %{QS:request})?(?:, upstream: “%{URI:upstream}”)?(?:, host: %{QS:host})?(?:, referrer: “%{URI:http_referer}”)?$
`
```
<187>Sep 24 16:10:46 jamingo.xyz.org nginx: 2018/09/24 16:10:46 [error] 94210#94210: *768531 FastCGI sent in stderr: "-web/releases/jamingo-andy-bli-opacweb-20180922174622/cacadenol/modules/database/classes/database/mysqli.php on line 52
PHP message: PHP Warning mysqli_connect(): (HY000/2002): Connection refused in file /var/www/jamingo-andy-bli-opacweb/releases/jamingo-andy-bli-opacweb-20180922174622/cacadenol/modules/database/classes/database/mysqli.php on line 52
PHP message: PHP Warning mysqli_connect(): (HY000/2002): Connection refused in file /var/www/jamingo-andy-bli-opacweb/releases/jamingo-andy-bli-opacweb-20180922174622/cacadenol/modules/database/classes/database/mysqli.php on line 52
PHP message: PHP Warning mysqli_connect(): (HY000/2002): Connection refused in file /var/www/jamingo-andy-bli-opacweb/releases/jamingo-andy-bli-opacweb-20180922174622/cacadenol/modules/database/classes/database/mysqli.php on line 52
PHP message: PHP Warning mysqli_connect(): (HY000/2002): Connection refused in file /var/www/jamingo-andy-bli-opacweb/releases/jamingo-andy-bli-opacweb-20180922174622/cacadenol/modules/database/classes/database/mysqli.php on line 52
PHP message: PHP Warning mysqli_connect(): (HY000/2002): Connection refused in file /var/www/jamingo-andy-bli-opacweb/releases/jamingo-andy-bli-opacweb-20180922174622/cacadenol/modules/database/classes/database/mysqli.php on line 52
PHP message: PHP Warning mysqli_connect(): (HY000/2002): Connection refused in file /var/www/jamingo-andy-bli-opacweb/releases/jamingo-andy-bli-opacweb-20180922174622/cacadenol/modules/database/classes/database/mysqli.php on line 52
PHP message: PHP Warning mysqli_connect(): (HY000/2002): Connection refused in file /var/www/jamingo-andy-bli-opacweb/releases/jamingo-andy-bli-opacweb-20180922174622/cacadenol/modules/database/classes/database/mysqli.php on line 52
PHP message: PHP Warning mysqli_connect(): (HY000/2002): Connection refused in file /var/www/jamingo-andy-bli-opacweb/releases/jamingo-andy-bli-opacweb-20180922174622/caca
```

this message is parsed very well without errors. but if we send a message from nginx error log with a length of 4096 bytes. this message is not processed, and there are too many errors in the logs graylog

```
2018-09-25T16:42:20.260+03:00 WARN [ProcessBuffer] Unable to process event MessageEvent{raw=null, message=null, messages=null}, sequence 285361587
java.lang.StackOverflowError: null
at java.util.regex.Pattern$GroupHead.match(Pattern.java:4660) ~[?:1.8.0_181]
at java.util.regex.Pattern$Loop.match(Pattern.java:4787) ~[?:1.8.0_181]
at java.util.regex.Pattern$GroupTail.match(Pattern.java:4719) ~[?:1.8.0_181]
at java.util.regex.Pattern$BranchConn.match(Pattern.java:4570) ~[?:1.8.0_181]
at java.util.regex.Pattern$CharProperty.match(Pattern.java:3779) ~[?:1.8.0_181]
at java.util.regex.Pattern$Branch.match(Pattern.java:4606) ~[?:1.8.0_181]
at java.util.regex.Pattern$GroupHead.match(Pattern.java:4660) ~[?:1.8.0_181]
at java.util.regex.Pattern$Loop.match(Pattern.java:4787) ~[?:1.8.0_181]
at java.util.regex.Pattern$GroupTail.match(Pattern.java:4719) ~[?:1.8.0_181]
at java.util.regex.Pattern$BranchConn.match(Pattern.java:4570) ~[?:1.8.0_181]
at java.util.regex.Pattern$CharProperty.match(Pattern.java:3779) ~[?:1.8.0_181]
at java.util.regex.Pattern$Branch.match(Pattern.java:4606) ~[?:1.8.0_181]
at java.util.regex.Pattern$GroupHead.match(Pattern.java:4660) ~[?:1.8.0_181]
at java.util.regex.Pattern$Loop.match(Pattern.java:4787) ~[?:1.8.0_181]
at java.util.regex.Pattern$GroupTail.match(Pattern.java:4719) ~[?:1.8.0_181]
at java.util.regex.Pattern$BranchConn.match(Pattern.java:4570) ~[?:1.8.0_181]
at java.util.regex.Pattern$CharProperty.match(Pattern.java:3779) ~[?:1.8.0_181]
at java.util.regex.Pattern$Branch.match(Pattern.java:4606) ~[?:1.8.0_181]
at java.util.regex.Pattern$GroupHead.match(Pattern.java:4660) ~[?:1.8.0_181]
at java.util.regex.Pattern$Loop.match(Pattern.java:4787) ~[?:1.8.0_181]
at java.util.regex.Pattern$GroupTail.match(Pattern.java:4719) ~[?:1.8.0_181]
at java.util.regex.Pattern$BranchConn.match(Pattern.java:4570) ~[?:1.8.0_181]
at java.util.regex.Pattern$CharProperty.match(Pattern.java:3779) ~[?:1.8.0_181]
at java.util.regex.Pattern$Branch.match(Pattern.java:4606) ~[?:1.8.0_181]
at java.util.regex.Pattern$GroupHead.match(Pattern.java:4660) ~[?:1.8.0_181]
```

this pattern multi line parsing (?(.|\r|\n)*) not work with 4096 bytes messages and buffer overflow occurs sequence 285361587 .

* Graylog Version: 2.4.6
* Elasticsearch Version: 5.6.12
* MongoDB Version: 3.2.11-2+deb9u1
* Operating System: Debian 9.5
* Browser version: Chrome,Opera, etc...

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.