dotnet / dotnet/machinelearning
ArgumentOutOfRangeException when running Microsoft.ML.Fairlearn.Tests.GridSearchTest.TestGridSearchTrialRunner2
- Dominant language
- C#
- Stars
- 9.4k
- Forks
- 2k
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 11
Description
## Build Information
Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=419700
Build error leg or test failing: Microsoft.ML.Fairlearn.Tests.WorkItemExecution
Pull request: https://github.com/dotnet/machinelearning/pull/6837
## Error Message
```
[Source=AutoMLExperiment, Kind=Trace] Channel started
[Source=AutoMLExperiment, Kind=Info] Update Running Trial - Id: 0
[Source=AutoMLExperiment, Kind=Trace] Microsoft.ML.AutoML.DefaultPerformanceMonitor has been started
[Source=AutoMLExperiment, Kind=Trace] current CPU: 0, current Memory(mb): 48.51171875
[Source=AutoMLExperiment, Kind=Trace] maximum memory usage: , PeakMemoryUsage: 48.51171875 trialIsCancelled: False
[Source=AutoMLExperiment, Kind=Trace] trial setting - {"TrialId":0,"StartedAtUtc":"2023-09-26T20:49:48.6702343Z","EndedAtUtc":null,"Parameter":{"_lambda_search_space":{"2_neg":-4.00442839,"2_pos":0.776618361,"3_neg":-2.88146996,"3_pos":8.18861198},"_pipeline_":{"_SCHEMA_":"e0 * e5","e0":{},"e1":{"NumberOfLeaves":1487,"MinimumExampleCountPerLeaf":23,"NumberOfTrees":61,"MaximumBinCountPerFeature":114,"FeatureFraction":0.4980983951658306,"LearningRate":1.1821894183157802E-07,"LabelColumnName":"y","FeatureColumnName":"Features","ExampleWeightColumnName":"signedWeight","DiskTranspose":false},"e2":{"NumberOfTrees":5,"NumberOfLeaves":41,"FeatureFraction":0.404329568,"LabelColumnName":"y","FeatureColumnName":"Features","ExampleWeightColumnName":"signedWeight"},"e3":{"NumberOfLeaves":17267,"MinimumExampleCountPerLeaf":63,"LearningRate":5.8468910203332519E-09,"NumberOfTrees":2221,"SubsampleFraction":2.8959196043297387E-08,"MaximumBinCountPerFeature":11,"FeatureFraction":0.88016485093703201,"L1Regularization":4.0092348571131609E-05,"L2Regularization":0.093939385599205991,"LabelColumnName":"y","FeatureColumnName":"Features","ExampleWeightColumnName":"signedWeight"},"e4":{"L1Regularization":9.83808041,"L2Regularization":4.45183802,"LabelColumnName":"y","FeatureColumnName":"Features","ExampleWeightColumnName":"signedWeight"},"e5":{"L1Regularization":80.4055481,"L2Regularization":2602.4187,"LabelColumnName":"y","FeatureColumnName":"Features","ExampleWeightColumnName":"signedWeight"}}}}
[Source=AutoMLExperiment, Kind=Trace] current CPU: 0.09375, current Memory(mb): 72.10546875
[Source=AutoMLExperiment, Kind=Trace] maximum memory usage: , PeakMemoryUsage: 72.10546875 trialIsCancelled: False
[Source=AutoMLExperiment, Kind=Trace]
Exception thrown during Trial 0 with configuration {"TrialId":0,"StartedAtUtc":"2023-09-26T20:49:48.6702343Z","EndedAtUtc":null,"Parameter":{"_lambda_search_space":{"2_neg":-4.00442839,"2_pos":0.776618361,"3_neg":-2.88146996,"3_pos":8.18861198},"_pipeline_":{"_SCHEMA_":"e0 * e5","e0":{},"e1":{"NumberOfLeaves":1487,"MinimumExampleCountPerLeaf":23,"NumberOfTrees":61,"MaximumBinCountPerFeature":114,"FeatureFraction":0.4980983951658306,"LearningRate":1.1821894183157802E-07,"LabelColumnName":"y","FeatureColumnName":"Features","ExampleWeightColumnName":"signedWeight","DiskTranspose":false},"e2":{"NumberOfTrees":5,"NumberOfLeaves":41,"FeatureFraction":0.404329568,"LabelColumnName":"y","FeatureColumnName":"Features","ExampleWeightColumnName":"signedWeight"},"e3":{"NumberOfLeaves":17267,"MinimumExampleCountPerLeaf":63,"LearningRate":5.8468910203332519E-09,"NumberOfTrees":2221,"SubsampleFraction":2.8959196043297387E-08,"MaximumBinCountPerFeature":11,"FeatureFraction":0.88016485093703201,"L1Regularization":4.0092348571131609E-05,"L2Regularization":0.093939385599205991,"LabelColumnName":"y","FeatureColumnName":"Features","ExampleWeightColumnName":"signedWeight"},"e4":{"L1Regularization":9.83808041,"L2Regularization":4.45183802,"LabelColumnName":"y","FeatureColumnName":"Features","ExampleWeightColumnName":"signedWeight"},"e5":{"L1Regularization":80.4055481,"L2Regularization":2602.4187,"LabelColumnName":"y","FeatureColumnName":"Features","ExampleWeightColumnName":"signedWeight"}}}}
Exception Details: AUC is not defined when there is no negative class in the data
Parameter name: NegSample
Abandoning Trial 0 and continue training.
[Source=AutoMLExperiment, Kind=Trace] AUC is not defined when there is no negative class in the data
Parameter name: NegSample at Microsoft.ML.Data.EvaluatorBase`1.AucAggregatorBase`1.ComputeWeightedAuc(Double& unweighted) in D:\a\_work\1\s\src\Microsoft.ML.Data\Evaluators\AucAggregator.cs:line 116
at Microsoft.ML.Data.BinaryClassifierEvaluator.Aggregator.Finish() in D:\a\_work\1\s\src\Microsoft.ML.Data\Evaluators\BinaryClassifierEvaluator.cs:line 705
at Microsoft.ML.Data.BinaryClassifierEvaluator.<>c__DisplayClass32_0.b__0(UInt32 stratColKey, ReadOnlyMemory`1 stratColVal, Aggregator agg) in D:\a\_work\1\s\src\Microsoft.ML.Data\Evaluators\BinaryClassifierEvaluator.cs:line 261
at Microsoft.ML.Data.EvaluatorBase`1.ProcessData(IDataView data, RoleMappedSchema schema, Func`2 activeColsIndices, TAgg aggregator, AggregatorDictionaryBase[] dictionaries) in D:\a\_work\1\s\src\Microsoft.ML.Data\Evaluators\EvaluatorBase.cs:line 183
at Microsoft.ML.Data.EvaluatorBase`1.Microsoft.ML.Data.IEvaluator.Evaluate(RoleMappedData data) in D:\a\_work\1\s\src\Microsoft.ML.Data\Evaluators\EvaluatorBase.cs:line 33
at Microsoft.ML.Data.BinaryClassifierEvaluator.Evaluate(IDataView data, String label, String score, String predictedLabel) in D:\a\_work\1\s\src\Microsoft.ML.Data\Evaluators\BinaryClassifierEvaluator.cs:line 951
at Microsoft.ML.Fairlearn.GridSearchTrailRunner.RunAsync(TrialSettings settings, CancellationToken ct) in D:\a\_work\1\s\src\Microsoft.ML.Fairlearn\Reductions\GridSearchTrialRunner.cs:line 89
at Microsoft.ML.AutoML.AutoMLExperiment.d__24.MoveNext() in D:\a\_work\1\s\src\Microsoft.ML.AutoML\AutoMLExperiment\AutoMLExperiment.cs:line 273
[Source=AutoMLExperiment, Kind=Info] Update Failed Trial - Id: 0 - Pipeline: Unknown=>SdcaLogisticRegressionBinary
```
Fill the error message using [step by step known issues guidance](https://github.com/dotnet/arcade/blob/main/Documentation/Projects/Build%20Analysis/KnownIssues.md#how-to-fill-out-a-known-issue-error-section).
```json
{
"ErrorMessage": "System.ArgumentOutOfRangeException : AUC is not defined when there is no negative class in the data",
"ErrorPattern": "",
"BuildRetry": false,
"ExcludeConsoleLog": true
}
```
### Known issue validation
**Build: :mag_right:** https://dev.azure.com/dnceng-public/public/_build/results?buildId=419700
**Error message validated:** `System.ArgumentOutOfRangeException : AUC is not defined when there is no negative class in the data`
**Result validation: :x:** Known issue did not match with the provided build.
**Validation performed at:** 9/27/2023 6:13:57 PM UTC
### Report
#### Summary
|24-Hour Hit Count|7-Day Hit Count|1-Month Count|
|---|---|---|
|0|0|0|
Contributor guide
Assessment
This issue has not been assessed yet.