Deserialization for PCA model seems to be broken. Failed to find a default value for "k"
- Dominant language
- Scala
- Stars
- 1.5k
- Forks
- 315
- PR merge metrics
- No merged PRs in 30d
Description
I have created a PCA model and able to serializeToBundle without any issues. However, after deserializeFromBundle and trying to transform, I am encountering below error:
py4j.protocol.Py4JJavaError: An error occurred while calling o278.transform.
: java.util.NoSuchElementException: Failed to find a default value for k
at org.apache.spark.ml.param.Params.$anonfun$getOrDefault$2(params.scala:756)
at scala.Option.getOrElse(Option.scala:121)
at org.apache.spark.ml.param.Params.getOrDefault(params.scala:756)
at org.apache.spark.ml.param.Params.getOrDefault$(params.scala:753)
at org.apache.spark.ml.PipelineStage.getOrDefault(Pipeline.scala:41)
at org.apache.spark.ml.param.Params.$(params.scala:762)
at org.apache.spark.ml.param.Params.$$(params.scala:762)
at org.apache.spark.ml.PipelineStage.$(Pipeline.scala:41)
at org.apache.spark.ml.feature.PCAParams.validateAndTransformSchema(PCA.scala:55)
at org.apache.spark.ml.feature.PCAParams.validateAndTransformSchema$(PCA.scala:51)
at org.apache.spark.ml.feature.PCAModel.validateAndTransformSchema(PCA.scala:122)
at org.apache.spark.ml.feature.PCAModel.transformSchema(PCA.scala:156)
at org.apache.spark.ml.PipelineStage.transformSchema(Pipeline.scala:71)
at org.apache.spark.ml.feature.PCAModel.transform(PCA.scala:146)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
at py4j.Gateway.invoke(Gateway.java:282)
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
at py4j.commands.CallCommand.execute(CallCommand.java:79)
at py4j.GatewayConnection.run(GatewayConnection.java:238)
at java.lang.Thread.run(Thread.java:750)
However, I have passed value of k while initialization the model and serializing it.
model = PCAModel(k=3, inputCol="features", outputCol="pca")
It seems to be very similar to the issue: https://github.com/combust/mleap/issues/481 where certain variables are missed to be set.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.