microsoft / microsoft/SynapseML

Params from parent java estimators aren't copied to python mmlspark models

Open
#582 4 comments 0 reactions 1 assignee Claimed by @imatiach-msft View on GitHub
area/lightgbm
Dominant language
Scala
Stars
5.2k
Forks
868
Avg merge
22h 9m
Merged PRs (30d)
45

Description

The java params for mmlspark estimators like LightGBMClassifier, etc. aren't copied over to the python instances.

This is related to this Jira ticket: [PySpark ML Models should contain Param values](https://jira.apache.org/jira/browse/SPARK-10931)

A [temporary fix](https://github.com/apache/spark/pull/17849) was added so that the params can be accessed using `getOrDefault` method. This does make it possible to pull in the params from mmlspark models, like LightGBMClassifier:
Screen Shot 2019-06-07 at 9 20 27 AM

Spark developers are planning on incrementally updating the pyspark API to use the appropriate getter and setter methods, and having the pyspark models define the params within themselves (see [SPARK-21812](https://issues.apache.org/jira/browse/SPARK-21812)). For example, [CountVectorizer](https://github.com/apache/spark/pull/16770/files) was updated in this fashion.

Since it's very useful to be able to access model parameters, I propose updating mmlspark models in a similar fashion. Happy to contribute to this effort.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.