combust / combust/mleap

Case Insensitivity on MLeap Models

Open
#713 3 comments 0 reactions 0 assignees View on GitHub
enhancement help wanted
Dominant language
Scala
Stars
1.5k
Forks
315
PR merge metrics
No merged PRs in 30d

Description

By default, if you train a PySparkML model with a dataframe that has uppercase column names, and then try to run an inference with the same column names but in lowercase, the prediction will fail. Is there a parameter or way to set case insensitivity on inference?

I see [this](https://github.com/combust/mleap/blob/536b18decc12b3ff4b7d40a4f51ba1a11890e57c/mleap-executor/src/main/scala/ml/combust/mleap/executor/Transform.scala#L39-L40) checking for a strict vs relaxed select of the leapframe which I assume is what I'm looking for. ~~How can I set that when serializing a PySpark Model to an MLeap Bundle?~~

Thanks!

Edit: I see that my second question was wrong - It comes from the transform function, not embedded into the Bundle itself. So when I call `model.transform(frame)` is there documentation on how to pass in the relaxedSelect option?

Edit: It seems like I'm incorrect on what relaxedSelect does. It seems to just "not throw an error on columns that don't exist" instead of being case insensitive. Is there a case insensitivity option?

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the linked Transform.scala code, especially the strict versus relaxed LeapFrame selection, and review the issue's notes about model.transform(frame) and relaxedSelect. Determine where inference column matching is defined and whether a case-insensitivity option can be specified; done means lowercase inference columns work for models trained with uppercase names without breaking existing behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, scala, spark
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.