dotnet / dotnet/machinelearning

Re-training an ImageClassificationTrainer

Open
#4,778 5 comments 0 reactions 0 assignees View on GitHub
area-Vision enhancement Priority:2 usability
Dominant language
C#
Stars
9.4k
Forks
2k
Avg merge
2d 20h
Merged PRs (30d)
11

Description

### System information

- **OS version**: Windows 10 Pro 18363
- **.NET version**: Core 2.1
- **Platform**: x64
- **ML.NET version**: 0.15-preview

### Issue

- **What did you do?**
I started working from [this example](https://docs.microsoft.com/en-us/dotnet/api/microsoft.ml.visioncatalog.imageclassification?view=ml-dotnet#Microsoft_ML_VisionCatalog_ImageClassification_Microsoft_ML_MulticlassClassificationCatalog_MulticlassClassificationTrainers_Microsoft_ML_Vision_ImageClassificationTrainer_Options_), to perform multiclass classification using transfer learning.
Training, saving, loading, and consuming the model works as intended.
Now I am trying to train further on that trained model (re-training), using these 2 steps:
* [Extract pre-trained model parameters](https://docs.microsoft.com/en-us/dotnet/machine-learning/how-to-guides/retrain-model-ml-net#re-train-model)
* [Re-train model](https://docs.microsoft.com/en-us/dotnet/machine-learning/how-to-guides/retrain-model-ml-net#re-train-model)

Similar to those steps, I came up with this piece of code:

```csharp
// Extract trained model parameters
ImageClassificationModelParameters originalParameters = ((ISingleFeaturePredictionTransformer)trainedModel).Model as ImageClassificationModelParameters;
// Retrain model
MulticlassPredictionTransformer trainedModelFurther = mlContext.MulticlassClassification.Trainers.ImageClassification().Fit(trainImages, originalParameters);
```

- **What happened?**
At the end of the last line, the method `Fit(IDataView, ImageClassificationModelParameters)` does not exist ([docs](https://docs.microsoft.com/en-us/dotnet/api/microsoft.ml.vision.imageclassificationtrainer?view=ml-dotnet#methods)).

- **What did you expect?**
* First of all, I expected a similar method as `Fit(IDataView, LinearModelParameters)` in `OnlineGradientDescentTrainer` ([docs](https://docs.microsoft.com/en-us/dotnet/api/microsoft.ml.trainers.onlinelineartrainer-2.fit?view=ml-dotnet#Microsoft_ML_Trainers_OnlineLinearTrainer_2_Fit_Microsoft_ML_IDataView_Microsoft_ML_Trainers_LinearModelParameters_)), `Fit(IDataView, MaximumEntropyModelParameters)` in `LbfgsMaximumEntropyMulticlassTrainer` ([docs](https://docs.microsoft.com/en-us/dotnet/api/microsoft.ml.trainers.lbfgsmaximumentropymulticlasstrainer.fit?view=ml-dotnet#Microsoft_ML_Trainers_LbfgsMaximumEntropyMulticlassTrainer_Fit_Microsoft_ML_IDataView_Microsoft_ML_Trainers_MaximumEntropyModelParameters_)), or other trainers to exist.
I believe the `ImageClassificationTrainer` makes use of an `LbfgsMaximumEntropyMulticlassTrainer` behind the scenes, to classify the extracted features from the images.

* Secondly (unrelated), the method `Fit(IDataView, IDataView)` in `ImageClassificationTrainer` seems unnecessary, as you can already specify the validation set (or a fraction of the training set to use for validation) in `ImageClassificationTrainer.Options`.

### Question

Is it possible to re-train this model on new data, maybe using some extra steps to convert the `ImageClassificationTrainer` to an `LbfgsMaximumEntropyMulticlassTrainer`? If not, when is this functionality expected?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.