Grok Filter and Nested Objects/Fields Ambiguity
- Dominant language
- Java
- Stars
- 14.9k
- Forks
- 3.5k
- Avg merge
- 19h 14m
- Merged PRs (30d)
- 63
Description
This may belong in the [logstash-plugins](https://github.com/logstash-plugins) repo, but I'll guess I'll start here.
I'm going through the process of changing my logstash config to be elasticsearch 2.0 compliant. This means replacing dot separated fields with nested fields inside of objects. I had a look through the issues and couldn't find anything exactly similar, which might mean my terminology is incorrect.
In my grok patterns I want to specify nested fields. I couldn't see anything in the grok documentation that says I couldn't specify nested fields inside grok patterns. I figured I should be able to do something like this:
```
grok {
match => ["body", "^%{USER:[radius][Log-Level]}:\s*(?<[radius][message]>.*)" ]
tag_on_failure => ["_grok-radius-log"]
}
```
So I decided I'd go ahead and convert my filters to for nested field allocation and running a config test resulted in:
```
./logstash -t -f /home/user/log-shipment/logstash/conf.d/radius/
Configuration OK
```
:+1: Great I'm good to roll.
However, when I try to run logstash with the nested fields inside the grok pattern matches to do some parsing of radius data, I get this error:
```
./logstash -f /home/user/log-shipment/logstash/conf.d/radius/
Settings: Default filter workers: 4
The error reported is:
invalid char in group name <[radius][message]>: /^(?(?:[a-zA-Z0-9._-]+)):\s*(?<[radius][message]>.*)/m
```
I'm kind of interpreting this error to mean that the nested fields are being treated as a regex character class, which kind of make sense in the world of grok. This is kind of a documentation problem and/or a problem with the grok filter test. I'm not sure how I'm meant to deal with this practically and it would be great to clear it up.
This is replicated in both logstash 1.5.6 and logstash 2.1.0.
Contributor guide
Assessment
This issue has not been assessed yet.