microsoft / microsoft/SynapseML

How to track lightgbm eval performance during and post training?

Open
#1,306 3 comments 1 reaction 0 assignees View on GitHub
area/lightgbm
Dominant language
Scala
Stars
5.2k
Forks
868
Avg merge
22h 9m
Merged PRs (30d)
45

Description

One can specify the eval data and eval metric for lightgbm via the ```validationIndicatorCol``` and ```metric``` parameters. How can one:
1. See the iteration by iteration evaluation metric evolution like one can with non-spark-lightgbm?
2. See the evaluation evolution information (from point 1) post model training. At least something like ```transformer.bestEvalScore(evalDataId, evalMetricId)```. Best would be something like ```tranformer.getEvalResults(evalDataId, evalMetricId)```. Of course, the ```evalDataId``` and ```evalMetricId``` need to have been assigned before the training started

AB#1984493

Contributor guide

Open the contributing guide

Research direction

Start by tracing how validationIndicatorCol and metric are handled during LightGBM training, then compare the available evaluation reporting with non-Spark LightGBM. Define how iteration-level results and post-training access through APIs such as bestEvalScore or getEvalResults should work, including evaluation identifiers. Done means the requested metrics are available during and after training with documented behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
scala
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.