Graylog2 / Graylog2/graylog2-server
Performance Improvement: Combine Regex in Sigma Rules
- Dominant language
- Java
- Stars
- 8.1k
- Forks
- 1.1k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 217
Description
Sigma rules are often slow to evaluate because they use a lot of "OR" terms. Using the regex-native OR operator leads to large performance improvements.
## What?
Sigma rules lead to queries containing many terms combined by OR. For example:
```
title: Process Command Line Contains Any Letter a-z
logsource:
category: process_creation
product: windows
detection:
selection:
CommandLine|contains:
- 'a'
- 'b'
- 'c'
...
- 'x'
- 'y'
- 'z'
condition: selection
```
Graylog translates this into:
```
process_command_line:(/.*a.*/ OR /.*b.*/ OR /.*c.*/ OR /.*d.*/ OR /.*e.*/ OR /.*f.*/
OR /.*g.*/ OR /.*h.*/ OR /.*i.*/ OR /.*j.*/ OR /.*k.*/
OR /.*l.*/ OR /.*m.*/ OR /.*n.*/ OR /.*o.*/ OR /.*p.*/
OR /.*q.*/ OR /.*r.*/ OR /.*s.*/ OR /.*t.*/ OR /.*u.*/
OR /.*v.*/ OR /.*w.*/ OR /.*x.*/ OR /.*y.*/ OR /.*z.*/)
```
A much more performant translation is this, however:
```
process_command_line:/.*(a|b|c|d|e|f|g|h|i|j|k|l|m|n|o|p|q|r|s|t|u|v|w|x|y|z).*/
```
## Why?
Time measurements:
OR-query: 14 seconds
regex query: 4 seconds
This matters for:
- general resource usage of Graylog
- every time an alert is investigated the analysts must wait for the query to load
## Your Environment
* Graylog Version: 7.1.0+549ae7f
* OpenSearch Version: graylog-datanode 7.1.0+549ae7f
* MongoDB Version: irrelevant
* Operating System: Linux
* Browser version: irrelevant
Contributor guide
Assessment
This issue has not been assessed yet.