combust / combust/mleap

After loading Mleap bundle returning org.apache.spark.ml.PipelineModel object instead of ml.combust.mleap.runtime.frame.Transformer

Open
#377 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Scala
Stars
1.5k
Forks
315
PR merge metrics
No merged PRs in 30d

Description

I am trying to use [mleap-xgboost-spark][1] library in my Java project and I am successfully able to train and save xgboost model.

In the below code I am trying to load the model back in memory so I can use it for prediction.
```

try(BundleFile bundleFile = BundleFile.apply("jar:file:/model.zip")) {
MleapContext context = MleapContext.apply(BundleRegistry$.MODULE$.apply("ml.combust.mleap.spark.registry.v22"));
Try> loadResult = MleapSupport$.MODULE$.MleapBundleFileOps(bundleFile)
.loadMleapBundle(context);

if (!(loadResult instanceof Success)) {
Throwable throwable = loadResult.failed().get();
throwable.printStackTrace();
}

System.out.println(loadResult.get().root().getClass());
}

```
Unfortunately/weirdly, `loadResult.get().root()` returning `org.apache.spark.ml.PipelineModel` object instead of `ml.combust.mleap.runtime.frame.Transformer`. Also, as per mleap library, `loadResult.get().root()` should return `ml.combust.mleap.runtime.frame.Transformer` so when try to use `getClass()` it throws me a `java.lang.ClassCastException: org.apache.spark.ml.PipelineModel cannot be cast to ml.combust.mleap.runtime.frame.Transformer`

Note, I explicitly pass "ml.combust.mleap.spark.registry.v22" bundle registry to MleapContext because `MleapContext.defaultContext()` not registering `xgboost.classifier -> ml.combust.mleap.xgboost.bundle.ops.XGBoostClassificationOp` the I required to load xgboost model back.

Please let me know if any other information is required.

[1]: https://github.com/combust/mleap/tree/master/mleap-xgboost-spark

Update:-

I guess Problem is in "ml.combust.mleap.spark.registry.v22" registry. Can someone suggest the way I can use the MleapContext where I can load the xgboost model and `loadResult.get().root()` return `ml.combust.mleap.runtime.frame.Transformer`?

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the Java loading snippet with BundleFile, MleapContext, BundleRegistry$.MODULE$.apply("ml.combust.mleap.spark.registry.v22"), and MleapSupport$.MODULE$.MleapBundleFileOps. Inspect the mleap-xgboost-spark registry and the loadMleapBundle result to determine why root() is an org.apache.spark.ml.PipelineModel. Done means the supported registry loads the XGBoost model and returns the expected runtime Transformer without the ClassCastException.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, scala, spark
Domain
backend, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.