Graylog2 / Graylog2/graylog2-server
Pipeline Regex Match Group with Escape Character issues
- Dominant language
- Java
- Stars
- 8.1k
- Forks
- 1.1k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 217
Description
i am using pipelines to parse a specific line of syslog information and use regex to parse through that information, using match groups to add specific information to custom fields
message example:
10.101.253.11 10.101.253.11 CONF-6-ARM: Rate switch 3761 Mbps,QAM32->4513 Mbps,QAM64 (L:-46, R:-50) From: 172.15.107.10
i have my regex setup thusly:
^[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3} [0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3} [A-Z]±[0-9]{1}-[A-Z]+: Rate switch ([0-9]+) [a-zA-Z]+,([a-zA-Z]+[0-9]+)->([0-9]+) [a-zA-Z]+,([a-zA-Z]+[0-9]+) (L:(-[0-9]+), R:(-[0-9]+)) From: [0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}$
this gives me the following match groups (per regex101.com)
|Group 1.|53-57|3761|
|Group 2.|63-68|QAM32|
|Group 3.|70-74|4513|
|Group 4.|80-85|QAM64|
|Group 5.|89-92|-46|
|Group 6.|96-99|-50|
My problem is when entering this into the pipeline rule i want to use to process this type of message i have problems when trying to escape ‘(’ and then use the information immediately after it to populate a match group i get multiple errors starting at “(L:-46” in my message string because that’s a character i need to escape in order to proceed to creating a match group for the information contained in it
pipeline rule:
rule “regex bridgewave modulation event”
when
regex("^[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3} [0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3} [A-Z]±[0-9]{1}-[A-Z]+: Rate switch ([0-9]+) [a-zA-Z]+,([a-zA-Z]+[0-9]+)->([0-9]+) [a-zA-Z]+,([a-zA-Z]+[0-9]+) (L:(-[0-9]+), R:(-[0-9]+)) From: [0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}$", to_string($message.message)).matches == true
then
let results = regex("^[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3} [0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3} [a-zA-Z]±[0-9]-ARM: Rate switch ([0-9]+) Mbps,(QAM[0-9]+)->([0-9]+) Mbps,(QAM[0-9]+) (L:-[0-9]+, R:-[0-9]+) From: [0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}$", to_string($message.message), [“from_speed_in_mpbs”, “from_qam_rate”, “to_speed_in_mpbs”, “to_qam_rate”]);
set_fields(results);
end
## Expected Behavior
i should be able to use '\' to escape the '(' character and then use another set of () to create my match group. I am getting errors as if there is a syntax issue but there is none
## Current Behavior

the attached errors when the regex syntax is correct
## Possible Solution
per thread https://community.graylog.org/t/pipeline-regex-match-group-with-escape-character-issues/7507 i was asked by staff member Jan Doberstein to create this bug report
"the problem here is that you need to escape the two fields differently. Di you mind opening a bug report over at https://github.com/Graylog2/graylog2-server/issues that the escaping should be the same for both fields?"
## Steps to Reproduce (for bugs)
1. use above test syslog message against regex in piepline rule. Pipeline rule displays errors that render it unable to be used
## Context
I'm attempting to use pipelines to parse data from syslog messages into custom fields
## Your Environment
* Graylog Version: 2.4.6+ceaa7e4, codename Wildwuchs
* Elasticsearch Version: Unknown
* MongoDB Version: Unknown
* Operating System: Windows 10
* Browser version: Chrome 70.0.3538.77
Contributor guide
Assessment
This issue has not been assessed yet.