Case Insensitivity on MLeap Models
- Dominant language
- Scala
- Stars
- 1.5k
- Forks
- 315
- PR merge metrics
- No merged PRs in 30d
Description
By default, if you train a PySparkML model with a dataframe that has uppercase column names, and then try to run an inference with the same column names but in lowercase, the prediction will fail. Is there a parameter or way to set case insensitivity on inference?
I see [this](https://github.com/combust/mleap/blob/536b18decc12b3ff4b7d40a4f51ba1a11890e57c/mleap-executor/src/main/scala/ml/combust/mleap/executor/Transform.scala#L39-L40) checking for a strict vs relaxed select of the leapframe which I assume is what I'm looking for. ~~How can I set that when serializing a PySpark Model to an MLeap Bundle?~~
Thanks!
Edit: I see that my second question was wrong - It comes from the transform function, not embedded into the Bundle itself. So when I call `model.transform(frame)` is there documentation on how to pass in the relaxedSelect option?
Edit: It seems like I'm incorrect on what relaxedSelect does. It seems to just "not throw an error on columns that don't exist" instead of being case insensitive. Is there a case insensitivity option?
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the linked Transform.scala code, especially the strict versus relaxed LeapFrame selection, and review the issue's notes about model.transform(frame) and relaxedSelect. Determine where inference column matching is defined and whether a case-insensitivity option can be specified; done means lowercase inference columns work for models trained with uppercase names without breaking existing behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, scala, spark
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100