split filter is not working as expected - split a json array/array
- Dominant language
- Java
- Stars
- 14.9k
- Forks
- 3.5k
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 88
Description
I have a JSON-Message with an array in an array. I want to split that into multiple events. This is my configuration:
```
input {
stdin {
codec => "json"
}
}
filter {
split {
field => "bean"
}
if [bean][method] {
split {
field => "bean[method]"
}
}
}
```
first array with 2 items, inner array 2 items:
```
{"foo": "someValue","bean": [
{"name": "bean1Name", "method": [{"name": "bean1meth1","max": 5},{"name": "bean1meth2","max": 3}]},
{"name": "bean2Name", "method": [{"name": "bean2meth1","max": 0},{"name": "bean2meth2","max": 1}]}]
}
```
all fine as I need it:
```
{"foo":"someValue","bean":{"name":"bean1Name","method":{"name":"bean1meth1","max":5}},"@version":"1",...
{"foo":"someValue","bean":{"name":"bean1Name","method":{"name":"bean1meth2","max":3}},"@version":"1",...
{"foo":"someValue","bean":{"name":"bean2Name","method":{"name":"bean2meth1","max":0}},"@version":"1",...
{"foo":"someValue","bean":{"name":"bean2Name","method":{"name":"bean2meth2","max":1}},"@version":"1",...
```
Now the problems, the first array has two items, inner array just one:
```
{"foo": "someValue","bean": [
{"name": "bean1Name", "method": [{"name": "bean1meth1","max": 5}]},
{"name": "bean2Name", "method": [{"name": "bean2meth1","max": 0}]}]
}
```
As you can see, the method is written as a list again. Not what I want:
```
{"foo":"someValue","bean":{"name":"bean1Name","method":[{"name":"bean1meth1","max":5}]},"@version":"1",...
{"foo":"someValue","bean":{"name":"bean2Name","method":[{"name":"bean2meth1","max":0}]},"@version":"1",...
```
next problem, the first array has one item:
```
{"foo": "someValue","bean": [
{"name": "bean1Name", "method": [{"name": "bean1meth1","max": 5}]}]
}
```
That is'n working at all:
```
Only String and Array types are splittable. field:bean[method] is of type = Java::JavaUtil::LinkedHashMap> ... split.rb:46:in `filter'
```
I use Logstash 1.5.2
Contributor guide
Research direction
Start with the nested split filter configuration and the reported split.rb:46 location. Run the three JSON examples through Logstash 1.5.2 and compare how singleton inner arrays are represented with the expected events. Done means nested arrays produce consistent object values instead of singleton lists or a type error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100