dotnet / dotnet/machinelearning
Fix Recursive Parsing for PFI
- Dominant language
- C#
- Stars
- 9.4k
- Forks
- 2k
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 11
Description
**System Information (please complete the following information):**
- Windows 11
- ML.NET Version 1.7, AutoML 0.19.0
- .NET Version: .NET Core 3.1
**Describe the bug**
ArgumentNullException while attempting to retrieve PFI
The model provided does not have a compatible predictor (Parameter 'lastTransformer')
Stack trace
at Microsoft.ML.Runtime.Contracts.CheckValue[T](IExceptionContext ctx, T val, String paramName, String msg)
at Microsoft.ML.PermutationFeatureImportanceExtensions.PermutationFeatureImportance[TMetric,TResult](IHostEnvironment env, ITransformer model, IDataView data, Func`1 resultInitializer, Func`2 evaluationFunc, Func`3 deltaFunc, Int32 permutationCount, Boolean useFeatureWeightFilter, Nullable`1 numberOfExamplesToUse)
at Microsoft.ML.PermutationFeatureImportanceExtensions.PermutationFeatureImportance(RegressionCatalog catalog, ITransformer model, IDataView data, String labelColumnName, Boolean useFeatureWeightFilter, Nullable`1 numberOfExamplesToUse, Int32 permutationCount)
**To Reproduce**
Not sure how best to create a minimal reproducible example including data, but here's the core of what I did (based on https://github.com/dotnet/machinelearning/pull/5934)
```
MLContext mlContext = new MLContext();
ColumnInferenceResults columnInference = mlContext.Auto().InferColumns(trainDataPath, "Label", groupColumns: false);
TextLoader textLoader = mlContext.Data.CreateTextLoader(columnInference.TextLoaderOptions);
IDataView trainDataView = textLoader.Load(trainDataPath);
IDataView testDataView = textLoader.Load(testDataPath);
IEstimator preFeaturizer =
mlContext.Transforms.Categorical.OneHotEncoding(
new T().GetOneHotInputColumnNames().Select(_ => new InputOutputColumnPair(_)).ToArray()
);
ColumnInformation columnInformation = columnInference.ColumnInformation;
columnInformation.IgnoredColumnNames.AddIfMissing("Foo");
columnInformation.CategoricalColumnNames.Remove("Foo");
columnInformation.NumericColumnNames.Remove("Foo");
BinaryExperimentSettings experimentSettings = new BinaryExperimentSettings()
{
MaxExperimentTimeInSeconds = experimentTime,
OptimizingMetric = BinaryClassificationMetric.F1Score
};
var experiment = mlContext.Auto().CreateBinaryClassificationExperiment(experimentSettings);
ExperimentResult experimentResult = experiment.Execute(trainDataView, columnInformation, preFeaturizer, progress);
RunDetail bestRun = experimentResult.BestRun;
// Exception thrown here
var permutationFeatureImportance = mlContext
.Regression
.PermutationFeatureImportance(bestRun.Model, testDataViewWithBestScore, permutationCount: 3);
```
The `LastTransformer` of `bestRun.Model` is `BinaryPredictionTransformer>`
**Expected behavior**
Retrieve the PFI information.
Contributor guide
Assessment
This issue has not been assessed yet.