microsoft / microsoft/SynapseML
How to track lightgbm eval performance during and post training?
- Dominant language
- Scala
- Stars
- 5.2k
- Forks
- 868
- Avg merge
- 22h 9m
- Merged PRs (30d)
- 45
Description
One can specify the eval data and eval metric for lightgbm via the ```validationIndicatorCol``` and ```metric``` parameters. How can one:
1. See the iteration by iteration evaluation metric evolution like one can with non-spark-lightgbm?
2. See the evaluation evolution information (from point 1) post model training. At least something like ```transformer.bestEvalScore(evalDataId, evalMetricId)```. Best would be something like ```tranformer.getEvalResults(evalDataId, evalMetricId)```. Of course, the ```evalDataId``` and ```evalMetricId``` need to have been assigned before the training started
AB#1984493
Contributor guide
Research direction
Start by tracing how validationIndicatorCol and metric are handled during LightGBM training, then compare the available evaluation reporting with non-Spark LightGBM. Define how iteration-level results and post-training access through APIs such as bestEvalScore or getEvalResults should work, including evaluation identifiers. Done means the requested metrics are available during and after training with documented behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- scala
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100