dotnet / dotnet/machinelearning
Get Loss During Training for Visualization (Learning Curve Graph)
- Dominant language
- C#
- Stars
- 9.4k
- Forks
- 2k
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 11
Description
**Is your feature request related to a problem? Please describe.**
I need a way to visualize how my model is learning during training, which is a comparison between training loss and test loss.
**Describe the solution you'd like**
An event handler that enables the ability to extract loss during training.
**Describe alternatives you've considered**
Running the model for x epochs, evaluating the model, then retraining the model in a loop. This unfortunately does not work for all models, such as LightGbm that can't be retrained.
```csharp
var kfold = ctx.BinaryClassification.CrossValidate(training, estimator, param.kfold);
var bestModel = kfold.OrderByDescending(p => p.Metrics.Accuracy).Select(p => p.Model).First();
var testOutput = bestModel.Transform(test);
var metrics = ctx.BinaryClassification.Evaluate(testOutput);
// This code doesn't work as estimator is IEstimator and bestModel is ITransform. Not sure how you would do this...
estimator = bestModel;
```
**Additional context**
Ultimately, I am trying to analyze what the models I'm comparing are actually doing and so far I haven't found any documentation or any straightforward way to do it.

Contributor guide
Assessment
This issue has not been assessed yet.