dotnet / dotnet/machinelearning

Cursor.MoveNext() gets stuck

Open
#5,926 2 comments 0 reactions 0 assignees View on GitHub
area-Core bug
Dominant language
C#
Stars
9.4k
Forks
2k
Avg merge
2d 20h
Merged PRs (30d)
11

Description

**System Information**
- OS & Version: Windows 10]
- ML.NET Version: ML.NET v1.6.0
- .NET Version: .NET 5.0

**Describe the bug**
After running auto.ml and using a resulting model to perform a transform on a DataView; sometimes the cursor created on the DataView gets "stuck" in the MoveNext call. The code is stuck in the _batchInputs.Take(); line as shown in the attached screenshot. This only seems to happen when the trained model is a **FastForestBinary.** Sometimes it runs just fine.

**To Reproduce**
Transform Input Data:
```
var transformPipeline = mlContext
.Transforms
.Concatenate("Features", featureColumns.Select(fc => fc.MlModelName).ToArray())
.Append(mlContext.Transforms.NormalizeMinMax("Features"))
.Append(mlContext.Transforms.SelectColumns(new[] { "Features", "Label" }));
```
Transform Data with AutoML model.
```
var linearModel = ((TransformerChain)trainedModel).LastTransformer;
var simpleScoredDataset = linearModel.Transform(transformedData);
```
Add CalculateFeatureContribution Transform to the results
```
FeatureContributionCalculatingEstimator featureContributionCalculatingEstimator;
if (linearModel is ISingleFeaturePredictionTransformer)
{
featureContributionCalculatingEstimator = mlContext.Transforms.CalculateFeatureContribution((ISingleFeaturePredictionTransformer)linearModel);
}
else
{
featureContributionCalculatingEstimator =
(FeatureContributionCalculatingEstimator)
(typeof(ExplainabilityCatalog)
.GetMethods()
.Where(m => m.Name.Equals("CalculateFeatureContribution") && m.IsGenericMethod && m.IsStatic).FirstOrDefault()
.MakeGenericMethod(linearModel.GetType().GenericTypeArguments[0].GenericTypeArguments)
.Invoke(null, new object[] { mlContext.Transforms, linearModel, 10, 10, true }));
}
var linearFeatureContributionCalculator = featureContributionCalculatingEstimator.Fit(simpleScoredDataset);
```
Try and use results
```
using (var scoringPipeline = transformer.Append(linearModel).Append(linearFeatureContributionCalculator))
{
var results = scoringPipeline.Transform(data);

var resultSchema = scoringPipeline.GetOutputSchema(data.Schema);

var modelData = mlModelResults.Cast().ToList();
var modelDataType = modelData[0].GetType();

var x = mlContext.Data.CreateEnumerable(results, true, ignoreMissingColumns: true).ToList();
```
...CreateEnumerable or results.GetRowCursor(resultSchema).MoveNext() will get stuck

**Expected behavior**
Cursor iterates over the resulting DataView

**Screenshots, Code, Sample Projects**
Code Paused in Microsoft.ML:
![image](https://user-images.githubusercontent.com/22535483/132410939-437b8f81-ff85-47a4-8723-62e6566eca18.png)
Code causing issue:
![image](https://user-images.githubusercontent.com/22535483/132412409-898da7b3-f713-4612-a2c1-a3ac6556a011.png)

**Additional context**
This also happens when using: mlContext.Data.CreateEnumerable

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.