After loading Mleap bundle returning org.apache.spark.ml.PipelineModel object instead of ml.combust.mleap.runtime.frame.Transformer
- Dominant language
- Scala
- Stars
- 1.5k
- Forks
- 315
- PR merge metrics
- No merged PRs in 30d
Description
I am trying to use [mleap-xgboost-spark][1] library in my Java project and I am successfully able to train and save xgboost model.
In the below code I am trying to load the model back in memory so I can use it for prediction.
```
try(BundleFile bundleFile = BundleFile.apply("jar:file:/model.zip")) {
MleapContext context = MleapContext.apply(BundleRegistry$.MODULE$.apply("ml.combust.mleap.spark.registry.v22"));
Try> loadResult = MleapSupport$.MODULE$.MleapBundleFileOps(bundleFile)
.loadMleapBundle(context);
if (!(loadResult instanceof Success)) {
Throwable throwable = loadResult.failed().get();
throwable.printStackTrace();
}
System.out.println(loadResult.get().root().getClass());
}
```
Unfortunately/weirdly, `loadResult.get().root()` returning `org.apache.spark.ml.PipelineModel` object instead of `ml.combust.mleap.runtime.frame.Transformer`. Also, as per mleap library, `loadResult.get().root()` should return `ml.combust.mleap.runtime.frame.Transformer` so when try to use `getClass()` it throws me a `java.lang.ClassCastException: org.apache.spark.ml.PipelineModel cannot be cast to ml.combust.mleap.runtime.frame.Transformer`
Note, I explicitly pass "ml.combust.mleap.spark.registry.v22" bundle registry to MleapContext because `MleapContext.defaultContext()` not registering `xgboost.classifier -> ml.combust.mleap.xgboost.bundle.ops.XGBoostClassificationOp` the I required to load xgboost model back.
Please let me know if any other information is required.
[1]: https://github.com/combust/mleap/tree/master/mleap-xgboost-spark
Update:-
I guess Problem is in "ml.combust.mleap.spark.registry.v22" registry. Can someone suggest the way I can use the MleapContext where I can load the xgboost model and `loadResult.get().root()` return `ml.combust.mleap.runtime.frame.Transformer`?
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reproducing the Java loading snippet with BundleFile, MleapContext, BundleRegistry$.MODULE$.apply("ml.combust.mleap.spark.registry.v22"), and MleapSupport$.MODULE$.MleapBundleFileOps. Inspect the mleap-xgboost-spark registry and the loadMleapBundle result to determine why root() is an org.apache.spark.ml.PipelineModel. Done means the supported registry loads the XGBoost model and returns the expected runtime Transformer without the ClassCastException.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, scala, spark
- Domain
- backend, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100