microsoft / microsoft/SynapseML

Access Training Metrics

Open
#691 2 comments 2 reactions 1 assignee View on GitHub

@imatiach-msft is already working on this.

Since Sep 16, 2019.

area/lightgbm enhancement high priority
Dominant language
Scala
Stars
5.2k
Forks
868
Avg merge
22h 9m
Merged PRs (30d)
45

Description

**Is your feature request related to a problem? Please describe.**
I'd like to be able to access the metrics produced during model training. Currently I only see metrics in the logs and need to run my own prediction + evaluation after model training, which seems wasteful.

**Describe the solution you'd like**
```python
ranker = LightGBMRanker()
model = ranker.fit(df)

# access summary statistics about the training
metric_history = model.summary.metrics['training']['ndcg@5']
print(metric_history) -> [.001, .01, .7]
```

**Additional context**
Having a `summary` field on the model returned by training is something other spark libraries have done. [XGBoost](https://github.com/dmlc/xgboost/blob/master/jvm-packages/xgboost4j-spark/src/main/scala/ml/dmlc/xgboost4j/scala/spark/XGBoostTrainingSummary.scala), [MLLib](https://spark.apache.org/docs/2.4.4/api/python/pyspark.ml.html#pyspark.ml.classification.LogisticRegressionSummary)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.