combust / combust/mleap

java.lang.ClassCastException: scala.None$ cannot be cast to java.lang.Double

Open
#357 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Scala
Stars
1.5k
Forks
315
PR merge metrics
No merged PRs in 30d

Description

Cast Error when transforming leapframe.
I'm using the imputer from mleap-spark-extension and some fields of my data are null. I init these structs as `ScalarType.Double.setNullable(true)`. But when I use the model to transform my generated leapframe, there comes the error.
```
java.lang.ClassCastException: scala.None$ cannot be cast to java.lang.Double
at ml.combust.mleap.runtime.transformer.feature.Imputer$$anonfun$1.apply(Imputer.scala:14)
at ml.combust.mleap.runtime.frame.Row$class.udfValue(Row.scala:241)
at ml.combust.mleap.runtime.frame.ArrayRow.udfValue(ArrayRow.scala:17)
at ml.combust.mleap.runtime.frame.Row$class.withValue(Row.scala:221)
at ml.combust.mleap.runtime.frame.ArrayRow.withValue(ArrayRow.scala:17)
at ml.combust.mleap.runtime.frame.DefaultLeapFrame$$anonfun$withColumn$1$$anonfun$apply$2$$anonfun$2.apply(DefaultLeapFrame.scala:54)
at ml.combust.mleap.runtime.frame.DefaultLeapFrame$$anonfun$withColumn$1$$anonfun$apply$2$$anonfun$2.apply(DefaultLeapFrame.scala:54)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at scala.collection.immutable.List.foreach(List.scala:392)
at scala.collection.generic.TraversableForwarder$class.foreach(TraversableForwarder.scala:35)
at scala.collection.mutable.ListBuffer.foreach(ListBuffer.scala:45)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
at scala.collection.AbstractTraversable.map(Traversable.scala:104)
at ml.combust.mleap.runtime.frame.DefaultLeapFrame$$anonfun$withColumn$1$$anonfun$apply$2.apply(DefaultLeapFrame.scala:54)
at ml.combust.mleap.runtime.frame.DefaultLeapFrame$$anonfun$withColumn$1$$anonfun$apply$2.apply(DefaultLeapFrame.scala:53)
at scala.util.Success$$anonfun$map$1.apply(Try.scala:237)
at scala.util.Try$.apply(Try.scala:192)
at scala.util.Success.map(Try.scala:237)
at ml.combust.mleap.runtime.frame.DefaultLeapFrame$$anonfun$withColumn$1.apply(DefaultLeapFrame.scala:52)
at ml.combust.mleap.runtime.frame.DefaultLeapFrame$$anonfun$withColumn$1.apply(DefaultLeapFrame.scala:49)
at scala.util.Success.flatMap(Try.scala:231)
at ml.combust.mleap.runtime.frame.DefaultLeapFrame.withColumn(DefaultLeapFrame.scala:48)
at ml.combust.mleap.runtime.frame.SimpleTransformer$class.transform(Transformer.scala:112)
at ml.combust.mleap.runtime.transformer.feature.Imputer.transform(Imputer.scala:11)
at ml.combust.mleap.runtime.transformer.Pipeline$$anonfun$transform$2$$anonfun$apply$1.apply(Pipeline.scala:28)
at ml.combust.mleap.runtime.transformer.Pipeline$$anonfun$transform$2$$anonfun$apply$1.apply(Pipeline.scala:28)
at scala.util.Success.flatMap(Try.scala:231)
at ml.combust.mleap.runtime.transformer.Pipeline$$anonfun$transform$2.apply(Pipeline.scala:28)
at ml.combust.mleap.runtime.transformer.Pipeline$$anonfun$transform$2.apply(Pipeline.scala:28)
at scala.collection.LinearSeqOptimized$class.foldLeft(LinearSeqOptimized.scala:124)
at scala.collection.immutable.List.foldLeft(List.scala:84)
at ml.combust.mleap.runtime.transformer.Pipeline.transform(Pipeline.scala:28)
at com.zhihu.saturn.service.impl.RecommendationImpl.getSlotCardRank(RecommendationImpl.scala:31)
at com.zhihu.saturn.thrift.RecommendationService$Processor$get_slot_card_rank.getResult(RecommendationService.java:181)
at com.zhihu.saturn.thrift.RecommendationService$Processor$get_slot_card_rank.getResult(RecommendationService.java:165)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at com.zhihu.tzone.server.TProcessorInvocationHandler.invoke(TProcessorInvocationHandler.java:55)
at com.sun.proxy.$Proxy19.process(Unknown Source)
at org.apache.thrift.TMultiplexedProcessor.process(TMultiplexedProcessor.java:123)
at com.zhihu.tzone.server.TZoneServlet.doPost(TZoneServlet.java:98)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:845)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:583)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:518)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
at java.lang.Thread.run(Thread.java:745)
```
Some column values are null as showed below.
```
null| 0.0| null| 0.0| 1.0| 1.0| 1.0| 1.0| 1.0| 1.0|
```

Is the imputer cannot handle none value? I suppose imputer is designed to fill null values.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.