dotnet / dotnet/machinelearning

CreateEnumerable code fails in Visual Studio FSI but works in .NET Interactive notebook?

Open
#6,353 10 comments 0 reactions 0 assignees View on GitHub
area-Core F# Priority:1
Dominant language
C#
Stars
9.4k
Forks
2k
Avg merge
2d 20h
Merged PRs (30d)
11

Description

**System Information (please complete the following information):**
- OS & Version: Windows 11
- ML.NET Version: Microsoft.ML.AutoML, 0.20.0-preview.22356.1
- .NET Version: failed on both 6.0.400 and 7.0.100-rc.1.22431.12

**Describe the bug**
I am trying to create an F# IEnumerable from predictions. The code works in .NET Interactive notebooks, but it does not work in Visual Studio 2022 17.3.2 via F# interactive. I have also tried pasting the code into F# interactive by running `dotnet fsi` at the command line, but I get the same error.

When I run it in a notebook, I get predictions as expected. When I run it in a FSX script file in Visual Studio, I get the error "Could not find column 'Predicted@' (Parameter 'Schema')", yet the schema has a parameter named "Predicted".

**To Reproduce**
Code
```fsharp
#i "nuget:https://pkgs.dev.azure.com/dnceng/public/_packaging/MachineLearning/nuget/v3/index.json"
#r "nuget: Microsoft.ML.AutoML, 0.20.0-preview.22356.1"

open System
open Microsoft.ML
open Microsoft.ML.Data

let mlContext = new MLContext()

[]
type Predictors =
{
Date: DateTime
Return: single
DividendPrice: single
}

let predictors =
[{ Date = DateTime(1872,2,1)
Return = 0.008688271977f
DividendPrice = 0.05482897535f };
{ Date = DateTime(1872,3,1)
Return = 0.03739754111f
DividendPrice = 0.05333504081f };
{ Date = DateTime(1872,4,1)
Return = 0.03229662776f
DividendPrice = 0.05175264552f};
{ Date = DateTime(1872,5,1)
Return = 0.004451415502f
DividendPrice = 0.05051480234f }]

let mlData = mlContext.Data.LoadFromEnumerable(predictors)

let modelFit =
let model =
EstimatorChain()
.Append(mlContext.Transforms.CopyColumns("Label","Return"))
.Append(mlContext.Transforms.Concatenate("Features","DividendPrice"))
.Append(mlContext.Regression.Trainers.Ols())
model.Fit(mlData)

[]
type Prediction =
{
Predicted: single
Date: DateTime
}

let pred = modelFit.Transform(mlData)
let renamePipeline = mlContext.Transforms.CopyColumns("Predicted","Score")
let renamed = renamePipeline.Fit(pred).Transform(pred)
mlContext.Data.CreateEnumerable(renamed,reuseRowObject=false)
```

### Output in .NET interactive:

index | Predicted | Date
-- | -- | --
0 | 0.022715298 | 1872-02-01 00:00:00Z
1 | 0.021365488 | 1872-03-01 00:00:00Z
2 | 0.019935748 | 1872-04-01 00:00:00Z
3 | 0.018817322 | 1872-05-01 00:00:00Z

### Output in Visual Studio:
```
System.ArgumentOutOfRangeException: Could not find column 'Predicted@' (Parameter 'Schema')
at Microsoft.ML.Data.TypedCursorable`1..ctor(IHostEnvironment env, IDataView data, Boolean ignoreMissingColumns, InternalSchemaDefinition schemaDefn)
at Microsoft.ML.Data.TypedCursorable`1.Create(IHostEnvironment env, IDataView data, Boolean ignoreMissingColumns, SchemaDefinition schemaDefinition)
at Microsoft.ML.Data.CursoringUtils.AsCursorable[TRow](IHostEnvironment env, IDataView data, Boolean ignoreMissingColumns, SchemaDefinition schemaDefinition)
at Microsoft.ML.PipeEngine`1..ctor(IHostEnvironment env, IDataView pipe, Boolean ignoreMissingColumns, SchemaDefinition schemaDefinition)
at Microsoft.ML.DataOperationsCatalog.CreateEnumerable[TRow](IDataView data, Boolean reuseRowObject, Boolean ignoreMissingColumns, SchemaDefinition schemaDefinition)
at .$FSI_0004.main@() in C:\test.fsx:line 52
Stopped due to error
```

Here is the schema
```fsharp
> renamed.Schema |> Seq.toArray
val it: DataViewSchema.Column[] =
[|Date@: DateTime {Annotations = ;
Index = 0;
IsHidden = false;
Name = "Date@";
Type = DateTime;}; Return@: Single {Annotations = ;
Index = 1;
IsHidden = false;
Name = "Return@";
Type = Single;};
DividendPrice@: Single {Annotations = ;
Index = 2;
IsHidden = false;
Name = "DividendPrice@";
Type = Single;}; Date: DateTime {Annotations = ;
Index = 3;
IsHidden = false;
Name = "Date";
Type = DateTime;};
Return: Single {Annotations = ;
Index = 4;
IsHidden = false;
Name = "Return";
Type = Single;};
DividendPrice: Single {Annotations = ;
Index = 5;
IsHidden = false;
Name = "DividendPrice";
Type = Single;}; Label: Single {Annotations = ;
Index = 6;
IsHidden = false;
Name = "Label";
Type = Single;};
Features: Vector {SlotNames} {Annotations = SlotNames;
Index = 7;
IsHidden = false;
Name = "Features";
Type = Vector;};
Score: Single {ScoreColumnSetId, ScoreColumnKind, ScoreValueKind}
{Annotations = ScoreColumnSetId, ScoreColumnKind, ScoreValueKind;
Index = 8;
IsHidden = false;
Name = "Score";
Type = Single;};
Predicted: Single {ScoreColumnSetId, ScoreColumnKind, ScoreValueKind}
{Annotations = ScoreColumnSetId, ScoreColumnKind, ScoreValueKind;
Index = 9;
IsHidden = false;
Name = "Predicted";
Type = Single;}|]
```

**Expected behavior**
I expect the code to work in fsharp interactive

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.