microsoft / microsoft/SynapseML
Access Training Metrics
@imatiach-msft is already working on this.
Since Sep 16, 2019.
- Dominant language
- Scala
- Stars
- 5.2k
- Forks
- 868
- Avg merge
- 22h 9m
- Merged PRs (30d)
- 45
Description
**Is your feature request related to a problem? Please describe.**
I'd like to be able to access the metrics produced during model training. Currently I only see metrics in the logs and need to run my own prediction + evaluation after model training, which seems wasteful.
**Describe the solution you'd like**
```python
ranker = LightGBMRanker()
model = ranker.fit(df)
# access summary statistics about the training
metric_history = model.summary.metrics['training']['ndcg@5']
print(metric_history) -> [.001, .01, .7]
```
**Additional context**
Having a `summary` field on the model returned by training is something other spark libraries have done. [XGBoost](https://github.com/dmlc/xgboost/blob/master/jvm-packages/xgboost4j-spark/src/main/scala/ml/dmlc/xgboost4j/scala/spark/XGBoostTrainingSummary.scala), [MLLib](https://spark.apache.org/docs/2.4.4/api/python/pyspark.ml.html#pyspark.ml.classification.LogisticRegressionSummary)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.