elastic / elastic/logstash

Improve error message: NumberFormatException when checking for an element but only a List is available

Open
#6,561 6 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
14.9k
Forks
3.5k
Avg merge
1d 4h
Merged PRs (30d)
88

Description

https://discuss.elastic.co/t/numberformatexception-stops-logstash-from-working/71973/7

For all general issues, please provide the following details for fast resolution:

- Version:
5.1.2

- Operating System:
Linux chell 4.7.0-0.bpo.1-amd64 #1 SMP Debian 4.7.5-1~bpo8+1 (2016-09-30) x86_64 GNU/Linux

- Config File (if you have sensitive info, please remove it):
```
if [taskExecution][input][bidderId] {
...
}
```

- Sample Data:
This works:
```
{
...
"properties": {
"taskExecution": {
"input": {
"bidderId": 1096,
"changeType": "RulesChanged"
}
}
}
```

This don't:
```
{
...
"properties": {
"taskExecution": {
"input": [
{
"accountId": 527,
"invoiceId": 3259,
"hasInvoiceNumberCounter": true,
"hasChanges": true
}
]
}
}
}
```

- Steps to Reproduce:
Document above not works, because the input contains a List.

```
[2017-01-18T10:56:16,553][DEBUG][logstash.inputs.beats ] config LogStash::Inputs::Beats/@client_inactivity_timeout = 60
deploy@apollo:/etc/logstash$ docker logs logstash-services.1.7ilmsb0gdld6wkpd0j309g6es
[2017-01-18T10:56:25,425][DEBUG][logstash.filters.json ] Running json filter {:event=>2017-01-18T08:16:46.209Z orion {"level":"DEBUG","message":"InvoicesWorkflow/anonymous-step-2: WaitingForExecution","log_source":"smarter.ecommerce.commons.spring.task.ServiceLogTaskRunnerPlugin","properties":{"service":{"name":"Whoop"},"workflowExecution":{"workflowId":"InvoicesWorkflow","id":"InvoicesWorkflow:regenerateInvoice:7b802029-1719-4dfc-ba06-41d45b591347","stepId":"anonymous-step-2"},"taskExecution":{"group":"InvoicesWorkflow","name":"anonymous-step-2","id":"taskRunner://smecTaskRunner/InvoicesWorkflow/anonymous-step-2/8591c1aa-72b7-4cae-a4ed-8dac1d146f49_4","createdAt":1484667736438,"status":"WaitingForExecution","endStatus":null,"priority":100,"startedAt":null,"finishedAt":null,"input":[{"accountId":527,"invoiceId":3259,"hasInvoiceNumberCounter":true,"hasChanges":true}],"lockKey":null,"output":null,"cancellation":null,"exception":null,"duration":null}},"createdAt":1484667736438}}
[2017-01-18T10:56:25,425][DEBUG][logstash.filters.json ] Event after json filter {:event=>2017-01-18T08:16:46.209Z orion InvoicesWorkflow/anonymous-step-2: WaitingForExecution}
[2017-01-18T10:56:25,425][DEBUG][logstash.util.decorators ] filters/LogStash::Filters::Mutate: adding value to field {"field"=>"status", "value"=>["%{[taskExecution][status]}"]}
[2017-01-18T10:56:25,425][DEBUG][logstash.util.decorators ] filters/LogStash::Filters::Mutate: adding value to field {"field"=>"title", "value"=>["%{[workflowExecution][workflowId]} - %{[workflowExecution][stepId]}"]}
[2017-01-18T10:56:25,425][DEBUG][logstash.filters.mutate ] filters/LogStash::Filters::Mutate: removing tag {:tag=>"[taskExecution][exception]"}
...
Exception in pipelineworker, the pipeline stopped processing new events, please check your filter configuration and restart Logstash.
...
Exception in thread "[main]>worker0" java.lang.NumberFormatException: For input string: "bidderId"
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Integer.parseInt(Integer.java:580)
at java.lang.Integer.parseInt(Integer.java:615)
at org.logstash.Accessors.fetch(Accessors.java:130)
at org.logstash.Accessors.get(Accessors.java:20)
at org.logstash.Event.getUnconvertedField(Event.java:160)
at org.logstash.ext.JrubyEventExtLibrary$RubyEvent.ruby_get_field(JrubyEventExtLibrary.java:113)
at org.logstash.ext.JrubyEventExtLibrary$RubyEvent$INVOKER$i$1$0$ruby_get_field.call(JrubyEventExtLibrary$RubyEvent$INVOKER$i$1$0$ruby_get_field.gen)
at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:168)
at org.jruby.ast.CallOneArgNode.interpret(CallOneArgNode.java:57)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
at org.jruby.ast.IfNode.interpret(IfNode.java:110)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
at org.jruby.evaluator.ASTInterpreter.INTERPRET_BLOCK(ASTInterpreter.java:112)
at org.jruby.runtime.Interpreted19Block.evalBlockBody(Interpreted19Block.java:206)
at org.jruby.runtime.Interpreted19Block.yield(Interpreted19Block.java:157)
at org.jruby.runtime.Block.yield(Block.java:142)
at org.jruby.RubyArray.eachCommon(RubyArray.java:1606)
at org.jruby.RubyArray.each(RubyArray.java:1613)
at org.jruby.RubyArray$INVOKER$i$0$0$each.call(RubyArray$INVOKER$i$0$0$each.gen)
at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:143)
at org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:154)
at org.jruby.ast.CallNoArgBlockNode.interpret(CallNoArgBlockNode.java:64)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
at org.jruby.evaluator.ASTInterpreter.INTERPRET_BLOCK(ASTInterpreter.java:112)
at org.jruby.runtime.Interpreted19Block.evalBlockBody(Interpreted19Block.java:206)
at org.jruby.runtime.Interpreted19Block.yield(Interpreted19Block.java:194)
at org.jruby.runtime.Interpreted19Block.call(Interpreted19Block.java:125)
at org.jruby.runtime.Block.call(Block.java:101)
at org.jruby.RubyProc.call(RubyProc.java:300)
at org.jruby.RubyProc.call19(RubyProc.java:281)
at org.jruby.RubyProc$INVOKER$i$0$0$call19.call(RubyProc$INVOKER$i$0$0$call19.gen)
at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:210)
at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:206)
at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:168)
at org.jruby.ast.CallOneArgNode.interpret(CallOneArgNode.java:57)
at org.jruby.ast.DAsgnNode.interpret(DAsgnNode.java:110)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
at org.jruby.evaluator.ASTInterpreter.INTERPRET_BLOCK(ASTInterpreter.java:112)
at org.jruby.runtime.Interpreted19Block.evalBlockBody(Interpreted19Block.java:206)
at org.jruby.runtime.Interpreted19Block.yield(Interpreted19Block.java:194)
at org.jruby.runtime.Interpreted19Block.call(Interpreted19Block.java:125)
at org.jruby.runtime.Block.call(Block.java:101)
at org.jruby.RubyProc.call(RubyProc.java:300)
at org.jruby.internal.runtime.methods.ProcMethod.call(ProcMethod.java:64)
at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:210)
at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:206)
at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:168)
...
```

Contributor guide

Open the contributing guide

Research direction

Start with Accessors.java and reproduce the reported field lookup using the configuration and sample event in the issue, where input is a List. Trace the fetch path that parses bidderId, then verify that the resulting diagnostic explains the mismatch instead of exposing NumberFormatException.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, ruby
Domain
backend
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.