Reported behavior (confusing/weird message) is a bug / needs more meaningful messages.
- Dominant language
- Java
- Stars
- 14.9k
- Forks
- 3.5k
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 88
Description
Please post all product and debugging questions on our [forum](https://discuss.elastic.co/c/logstash). Your questions will reach our wider community members there, and if we confirm that there is a bug, then we can open a new issue here.
Logstash Plugins are located in a different organization: https://github.com/logstash-plugins. For bugs on specific Logstash plugins, for example, if Redis Output has a defect, please open it in the respective Redis Output repository.
For all general issues, please provide the following details for fast resolution:
- Version: Logstash 5.1
- Operating System: RHEL 6.6
- Config File (if you have sensitive info, please remove it):
- Sample Data:
- Steps to Reproduce: Please look at these steps we’ve taken and the exception it generates.
The following config file was used:
```
input {
stdin {
# codec => json
}
}
filter {
# strategy { symbol: xyz parameter { tag: 9001 value: 20170512-13:30:00 } parameter { tag: 9002 value: 20170512-20:00:00 } parameter { tag: 13032 value: 0 } parameter { tag: 9063 value: 0.0 } parameter { tag: 9062 value: 6717.0 } parameter { tag: 9054 value: Hidden_Dark } parameter { tag: 9064 value: 0.0 } parameter { tag: 7128 value: 1494595800000 } parameter { tag: 7129 value: 1494619200000 } parameter { tag: 28079 value: LNDARK } parameter { tag: 13031 value: 100 } parameter { tag: 7148 value: Hidden_Dark } } }
# ADDED FOR TESTING TO SIMULATE 'event' FIELD
grok {
match => {
"message" =>
"^%{GREEDYDATA:event}$"
}
}
grok {
match => {
"event" =>
"strategy { symbol: \S+ %{DATA:strategy_parameters} } }"
}
}
# kv is very limited, we have to make the parsable string as limited
# as kv itself
mutate {
gsub => [
"strategy_parameters", 'parameter \{ tag: ', ' ',
"strategy_parameters", ' value: ', ':',
"strategy_parameters", ' \} ', ''
]
}
# ready for kv
kv {
source => "strategy_parameters"
target => "strategy_parameters"
prefix => "tag_"
field_split => " "
value_split => ":"
}
mutate {
# rename => {"strategy_parameters" => "[event][strategy_parameters]"}
rename => {"strategy_parameters" => "[new_event][strategy_parameters]"}
}
}
output {
stdout { codec => rubydebug }
}
```
We think the crux of the problem is that the customer is carrying around a field called `event` that he continued to parse/extract from - finally getting to the point where he extracted a field called `strategy_parameters` and then re-writes that field by parsing it further using the key-value (kv) function. From here, he tried to assign that new `strategy_parameters` object back into the event field which still has whatever the original value was. We think this is what lines up with the beginning of the stacktrace:
```
Exception in thread "[main]>worker3" java.lang.ClassCastException: expecting List or Map, found class org.logstash.bivalues.StringBiValue
```
LS doesn't like that this new object is being added to the `event` object because they don't match, etc.
In our example, if we leave it as being reassigned to `event` if fails in 5.1 and 5.4 ... but if we add it to a new field called `new_event` then there is no issue - so the suggestion given to the customer would be to use a new field altogether or probably reset/dump `event` if he no longer needs it before.
We know the problem, we know the workaround so the question is now if the feature was designed with such a constraint in mind, or if the behavior is unintended? Or is this a bug?
The outcome here is:
> As it turned out - this is by design in the new Java based event - we got big speed improvements > as well as the binary serializing for the persistent queue but we lost the flexible type system of Ruby.
The issue and ask here is:
> The reported behavior (confusing/weird message) is a bug. The failure is expected, but the message is not useful to users.
Contributor guide
Assessment
This issue has not been assessed yet.