PR Validation Flaky Test

Open

#6,684 opened on May 15, 2023 1 comment 0 reactions 0 assignees View on GitHub

area-Trainers

Dominant language: C#
Stars: 9.4k
Forks: 2k
Avg merge: 2d 20h
Merged PRs (30d): 11

Description

Microsoft.ML.RunTests.TestPredictors.MulticlassLRTest is flaky on arm/arm64 devices. Disabling the test for now on non x86/x64 devices until we figure out the issue.

```
Microsoft.ML.RunTests.TestPredictors.MulticlassLRTest [FAIL]
[31;1m[m[37m Assert.Equal() Failure
[m[37m Expected: 0
[m[37m Actual: 2
[m[30;1m Stack Trace:
[m[37m /__w/1/s/test/Microsoft.ML.TestFramework/BaseTestBaseline.cs(201,0): at Microsoft.ML.RunTests.BaseTestBaseline.Done()
[m[37m /__w/1/s/test/Microsoft.ML.Predictor.Tests/TestPredictors.cs(156,0): at Microsoft.ML.RunTests.TestPredictors.MulticlassLRTest()
[m[30;1m Output:
[m[37m Running 'MulticlassLogisticRegression' on 'iris'
[m[37m Running as: TrainTest tr=MulticlassLogisticRegression{l1=0.001 l2=0.1 ot=1e-3 nt=1} data=/root/helix/work/correlation/test/data/iris.txt seed=1 test=/root/helix/work/correlation/test/data/iris.txt xf=Term{col=Label} out={/root/helix/work/workitem/e/TestOutput/MulticlassLogisticRegression/MulticlassLogisticRegression-TrainTest-iris-model.zip} dout={/root/helix/work/workitem/e/TestOutput/MulticlassLogisticRegression/MulticlassLogisticRegression-TrainTest-iris.txt} norm=no
[m[37m maml.exe TrainTest test=/root/helix/work/correlation/test/data/iris.txt tr=MulticlassLogisticRegression{l1=0.001 l2=0.1 ot=1e-3 nt=1} norm=No dout=/root/helix/work/workitem/e/TestOutput/MulticlassLogisticRegression/MulticlassLogisticRegression-TrainTest-iris.txt data=/root/helix/work/correlation/test/data/iris.txt out=/root/helix/work/workitem/e/TestOutput/MulticlassLogisticRegression/MulticlassLogisticRegression-TrainTest-iris-model.zip seed=1 xf=Term{col=Label}
[m[37m
[m[37m
[m[37m Not adding a normalizer.
[m[37m
[m[37m
[m[37m Beginning optimization
[m[37m
[m[37m
[m[37m num vars: 15
[m[37m
[m[37m
[m[37m improvement criterion: Mean Improvement
[m[37m
[m[37m
[m[37m L1 regularization selected 15 of 15 weights.
[m[37m
[m[37m
[m[37m Not training a calibrator because it is not needed.
[m[37m
[m[37m
[m[37m
[m[37m
[m[37m Confusion table
[m[37m
[m[37m
[m[37m ||========================
[m[37m
[m[37m
[m[37m PREDICTED || 0 | 1 | 2 | Recall
[m[37m
[m[37m
[m[37m TRUTH ||========================
[m[37m
[m[37m
[m[37m 0 || 50 | 0 | 0 | 1.0000
[m[37m
[m[37m
[m[37m 1 || 0 | 48 | 2 | 0.9600
[m[37m
[m[37m
[m[37m 2 || 0 | 1 | 49 | 0.9800
[m[37m
[m[37m
[m[37m ||========================
[m[37m
[m[37m
[m[37m Precision ||1.0000 |0.9796 |0.9608 |
[m[37m
[m[37m
[m[37m Accuracy(micro-avg): 0.980000
[m[37m
[m[37m
[m[37m Accuracy(macro-avg): 0.980000
[m[37m
[m[37m
[m[37m Log-loss: 0.072171
[m[37m
[m[37m
[m[37m Log-loss reduction: 0.934307
[m[37m
[m[37m
[m[37m
[m[37m
[m[37m OVERALL RESULTS
[m[37m
[m[37m
[m[37m ---------------------------------------
[m[37m
[m[37m
[m[37m Accuracy(micro-avg): 0.980000 (0.0000)
[m[37m
[m[37m
[m[37m Accuracy(macro-avg): 0.980000 (0.0000)
[m[37m
[m[37m
[m[37m Log-loss: 0.072171 (0.0000)
[m[37m
[m[37m
[m[37m Log-loss reduction: 0.934307 (0.0000)
[m[37m
[m[37m
[m[37m
[m[37m
[m[37m ---------------------------------------
[m[37m
[m[37m
[m[37m Physical memory usage(MB): 120
[m[37m
[m[37m
[m[37m Virtual memory usage(MB): 3549
[m[37m
[m[37m
[m[37m 5/15/2023 5:29:00 PM Time elapsed(s): 0.028
[m[37m
[m[37m
[m[37m
[m[37m
[m[37m Comparing /root/helix/work/workitem/e/TestOutput/MulticlassLogisticRegression/MulticlassLogisticRegression-TrainTest-iris-out.txt and /root/helix/work/correlation/test/BaselineOutput/Common/MulticlassLogisticRegression/MulticlassLogisticRegression-TrainTest-iris-out.txt
[m[37m Output matches baseline: 'MulticlassLogisticRegression/MulticlassLogisticRegression-TrainTest-iris-out.txt'
[m[37m Comparing /root/helix/work/workitem/e/TestOutput/MulticlassLogisticRegression/MulticlassLogisticRegression-TrainTest-iris.txt and /root/helix/work/correlation/test/BaselineOutput/Common/MulticlassLogisticRegression/netcoreapp/MulticlassLogisticRegression-TrainTest-iris.txt
[m[37m Output matches baseline: 'MulticlassLogisticRegression/MulticlassLogisticRegression-TrainTest-iris.txt'
[m[37m maml.exe Test dout=/root/helix/work/workitem/e/TestOutput/MulticlassLogisticRegression/MulticlassLogisticRegression-TrainTest-iris.txt data=/root/helix/work/correlation/test/data/iris.txt in=/root/helix/work/workitem/e/TestOutput/MulticlassLogisticRegression/MulticlassLogisticRegression-TrainTest-iris-model.zip seed=1
[m[37m
[m[37m
[m[37m
[m[37m
[m[37m Confusion table
[m[37m
[m[37m
[m[37m ||========================
[m[37m
[m[37m
[m[37m PREDICTED || 0 | 1 | 2 | Recall
[m[37m
[m[37m
[m[37m TRUTH ||========================
[m[37m
[m[37m
[m[37m 0 || 50 | 0 | 0 | 1.0000
[m[37m
[m[37m
[m[37m 1 || 0 | 48 | 2 | 0.9600
[m[37m
[m[37m
[m[37m 2 || 0 | 1 | 49 | 0.9800
[m[37m
[m[37m
[m[37m ||========================
[m[37m
[m[37m
[m[37m Precision ||1.0000 |0.9796 |0.9608 |
[m[37m
[m[37m
[m[37m Accuracy(micro-avg): 0.980000
[m[37m
[m[37m
[m[37m Accuracy(macro-avg): 0.980000
[m[37m
[m[37m
[m[37m Log-loss: 0.072171
[m[37m
[m[37m
[m[37m Log-loss reduction: 0.934307
[m[37m
[m[37m
[m[37m
[m[37m
[m[37m OVERALL RESULTS
[m[37m
[m[37m
[m[37m ---------------------------------------
[m[37m
[m[37m
[m[37m Accuracy(micro-avg): 0.980000 (0.0000)
[m[37m
[m[37m
[m[37m Accuracy(macro-avg): 0.980000 (0.0000)
[m[37m
[m[37m
[m[37m Log-loss: 0.072171 (0.0000)
[m[37m
[m[37m
[m[37m Log-loss reduction: 0.934307 (0.0000)
[m[37m
[m[37m
[m[37m
[m[37m
[m[37m ---------------------------------------
[m[37m
[m[37m
[m[37m Physical memory usage(MB): 120
[m[37m
[m[37m
[m[37m Virtual memory usage(MB): 3549
[m[37m
[m[37m
[m[37m 5/15/2023 5:29:00 PM Time elapsed(s): 0.008
[m[37m
[m[37m
[m[37m
[m[37m
[m[37m Suffix of length 27 compared against sequence of length 34
[m[37m Running 'MulticlassLogisticRegression' on 'iris'
[m[37m Running as: CV tr=MulticlassLogisticRegression{l1=0.001 l2=0.1 ot=1e-3 nt=1} data=/root/helix/work/correlation/test/data/iris.txt seed=1 xf=Term{col=Label} dout={/root/helix/work/workitem/e/TestOutput/MulticlassLogisticRegression/MulticlassLogisticRegression-CV-iris.txt} norm=no threads-
[m[37m maml.exe CV tr=MulticlassLogisticRegression{l1=0.001 l2=0.1 ot=1e-3 nt=1} threads=- norm=No dout=/root/helix/work/workitem/e/TestOutput/MulticlassLogisticRegression/MulticlassLogisticRegression-CV-iris.txt data=/root/helix/work/correlation/test/data/iris.txt seed=1 xf=Term{col=Label}
[m[37m
[m[37m
[m[37m Not adding a normalizer.
[m[37m
[m[37m
[m[37m Not adding a normalizer.
[m[37m
[m[37m
[m[37m Beginning optimization
[m[37m
[m[37m
[m[37m num vars: 15
[m[37m
[m[37m
[m[37m improvement criterion: Mean Improvement
[m[37m
[m[37m
[m[37m L1 regularization selected 15 of 15 weights.
[m[37m
[m[37m
[m[37m Not training a calibrator because it is not needed.
[m[37m
[m[37m
[m[37m Physical memory usage(MB): 120
[m[37m
[m[37m
[m[37m Virtual memory usage(MB): 3549
[m[37m
[m[37m
[m[37m 5/15/2023 5:29:00 PM Time elapsed(s): 0.018
[m[37m
[m[37m
[m[37m
[m[37m
[m[37m ***** Unexpected failure. Please refer to https://aka.ms/MLNetIssue to file an issue with details *****
[m[37m
[m[37m
[m[37m ***** Error log has been saved to '/tmp/TLC/Error_20230515_172900_e06f381a-3a62-4df3-b493-09ff5a4df87d.log', please refer to https://aka.ms/MLNetIssue to file an issue with details *****
[m[37m
[m[37m
[m[37m ===== Begin detailed dump =====
[m[37m
[m[37m
[m[37m (1) Unexpected exception: One or more errors occurred. (Assert failed: Assertion Failed
[m[37m
[m[37m
[m[37m Expected: True
[m[37m
[m[37m
[m[37m Actual: False), 'System.AggregateException'
[m[37m
[m[37m
[m[37m at System.Threading.Tasks.Task.WaitAllCore(Task[] tasks, Int32 millisecondsTimeout, CancellationToken cancellationToken)
[m[37m
[m[37m
[m[37m at System.Threading.Tasks.Task.WaitAll(Task[] tasks)
[m[37m
[m[37m
[m[37m at Microsoft.ML.Data.CrossValidationCommand.FoldHelper.GetCrossValidationTasks() in /__w/1/s/src/Microsoft.ML.Data/Commands/CrossValidationCommand.cs:line 471
[m[37m
[m[37m
[m[37m at Microsoft.ML.Data.CrossValidationCommand.RunCore(IChannel ch, String cmd) in /__w/1/s/src/Microsoft.ML.Data/Commands/CrossValidationCommand.cs:line 202
[m[37m
[m[37m
[m[37m at Microsoft.ML.Data.CrossValidationCommand.Run() in /__w/1/s/src/Microsoft.ML.Data/Commands/CrossValidationCommand.cs:line 134
[m[37m
[m[37m
[m[37m at Microsoft.ML.Tools.Maml.MainCore(IHostEnvironment env, String args, Boolean alwaysPrintStacktrace) in /__w/1/s/src/Microsoft.ML.Maml/MAML.cs:line 130
[m[37m
[m[37m
[m[37m (2) Unexpected exception: Assert failed: Assertion Failed
[m[37m
[m[37m
[m[37m Expected: True
[m[37m
[m[37m
[m[37m Actual: False, 'Xunit.Sdk.TrueException'
[m[37m
[m[37m
[m[37m at Xunit.Assert.True(Nullable`1 condition, String userMessage) in /_/src/xunit.assert/Asserts/BooleanAsserts.cs:line 132
[m[37m
[m[37m
[m[37m at Xunit.Assert.True(Boolean condition, String userMessage) in /_/src/xunit.assert/Asserts/BooleanAsserts.cs:line 116
[m[37m
[m[37m
[m[37m at Microsoft.ML.Internal.Internallearn.Test.GlobalBase.AssertHandler(String msg, IExceptionContext ectx) in /__w/1/s/test/Microsoft.ML.TestFramework/GlobalBase.cs:line 80
[m[37m
[m[37m
[m[37m at Microsoft.ML.Runtime.Contracts.DbgFailCore(String msg, IExceptionContext ctx) in /__w/1/s/src/Microsoft.ML.Core/Utilities/Contracts.cs:line 781
[m[37m
[m[37m
[m[37m at Microsoft.ML.Runtime.Contracts.DbgFail(IExceptionContext ctx) in /__w/1/s/src/Microsoft.ML.Core/Utilities/Contracts.cs:line 790
[m[37m
[m[37m
[m[37m at Microsoft.ML.Runtime.Contracts.Assert(IExceptionContext ctx, Boolean f) in /__w/1/s/src/Microsoft.ML.Core/Utilities/Contracts.cs:line 845
[m[37m
[m[37m
[m[37m at Microsoft.ML.Data.CacheDataView.ColumnCache.ImplVec`1.Fetch(Int32 idx, VBuffer`1& value) in /__w/1/s/src/Microsoft.ML.Data/DataView/CacheDataView.cs:line 1389
[m[37m
[m[37m
[m[37m at Microsoft.ML.Data.CacheDataView.RowCursor`1.<>c__DisplayClass8_0`1.b__0(TValue& value) in /__w/1/s/src/Microsoft.ML.Data/DataView/CacheDataView.cs:line 496
[m[37m
[m[37m
[m[37m at Microsoft.ML.Trainers.FeatureFloatVectorCursor.Accept() in /__w/1/s/src/Microsoft.ML.Data/Training/TrainerUtils.cs:line 855
[m[37m
[m[37m
[m[37m at Microsoft.ML.Trainers.FloatLabelCursor.Accept() in /__w/1/s/src/Microsoft.ML.Data/Training/TrainerUtils.cs:line 928
[m[37m
[m[37m
[m[37m at Microsoft.ML.Trainers.TrainingCursorBase.MoveNext() in /__w/1/s/src/Microsoft.ML.Data/Training/TrainerUtils.cs:line 555
[m[37m
[m[37m
[m[37m at Microsoft.ML.Trainers.LbfgsTrainerBase`3.TrainCore(IChannel ch, RoleMappedData data) in /__w/1/s/src/Microsoft.ML.StandardTrainers/Standard/LogisticRegression/LbfgsPredictorBase.cs:line 605
[m[37m
[m[37m
[m[37m at Microsoft.ML.Trainers.LbfgsTrainerBase`3.TrainModelCore(TrainContext context) in /__w/1/s/src/Microsoft.ML.StandardTrainers/Standard/LogisticRegression/LbfgsPredictorBase.cs:line 458
[m[37m
[m[37m
[m[37m at Microsoft.ML.Trainers.TrainerEstimatorBase`2.Microsoft.ML.ITrainer.Train(TrainContext context) in /__w/1/s/src/Microsoft.ML.Data/Training/TrainerEstimatorBase.cs:line 100
[m[37m
[m[37m
[m[37m at Microsoft.ML.Trainers.TrainerEstimatorBase`2.Microsoft.ML.ITrainer.Train(TrainContext context) in /__w/1/s/src/Microsoft.ML.Data/Training/TrainerEstimatorBase.cs:line 168
[m[37m
[m[37m
[m[37m at Microsoft.ML.Data.TrainUtils.TrainCore(IHostEnvironment env, IChannel ch, RoleMappedData data, ITrainer trainer, RoleMappedData validData, IComponentFactory`1 calibrator, Int32 maxCalibrationExamples, Nullable`1 cacheData, IPredictor inputPredictor, RoleMappedData testData) in /__w/1/s/src/Microsoft.ML.Data/Commands/TrainCommand.cs:line 280
[m[37m
[m[37m
[m[37m at Microsoft.ML.Data.TrainUtils.Train(IHostEnvironment env, IChannel ch, RoleMappedData data, ITrainer trainer, RoleMappedData validData, IComponentFactory`1 calibrator, Int32 maxCalibrationExamples, Nullable`1 cacheData, IPredictor inputPredictor, RoleMappedData testData) in /__w/1/s/src/Microsoft.ML.Data/Commands/TrainCommand.cs:line 255
[m[37m
[m[37m
[m[37m at Microsoft.ML.Data.CrossValidationCommand.FoldHelper.RunFold(Int32 fold) in /__w/1/s/src/Microsoft.ML.Data/Commands/CrossValidationCommand.cs:line 523
[m[37m
[m[37m
[m[37m at Microsoft.ML.Data.CrossValidationCommand.FoldHelper.<>c__DisplayClass24_0.b__0() in /__w/1/s/src/Microsoft.ML.Data/Commands/CrossValidationCommand.cs:line 463
[m[37m
[m[37m
[m[37m at System.Threading.Tasks.Task`1.InnerInvoke()
[m[37m
[m[37m
[m[37m at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
[m[37m
[m[37m
[m[37m --- End of stack trace from previous location ---
[m[37m
[m[37m
[m[37m at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)
[m[37m
[m[37m
[m[37m ====== End detailed dump =====
[m[37m
[m[37m
[m[37m *** Predictor returned -2
[m[37m Comparing /root/helix/work/workitem/e/TestOutput/MulticlassLogisticRegression/MulticlassLogisticRegression-CV-iris-out.txt and /root/helix/work/correlation/test/BaselineOutput/Common/MulticlassLogisticRegression/netcoreapp/MulticlassLogisticRegression-CV-iris-out.txt
[m[37m *** Failure #1: Output and baseline mismatch at line 3, expected 'Beginning optimization' but got 'Not adding a normalizer.' : 'MulticlassLogisticRegression/MulticlassLogisticRegression-CV-iris-out.txt'
[m[37m Void Fail(System.String, System.Object[]) /__w/1/s/test/Microsoft.ML.TestFramework/BaseTestBaseline.cs 224
[m[37m Boolean CheckEqualityFromPathsCore(System.String, System.String, System.String, Int32, Int32, NumberParseOption) /__w/1/s/test/Microsoft.ML.TestFramework/BaseTestBaseline.cs 546
[m[37m Boolean CheckEqualityCore(System.String, System.String, System.String, Boolean, Int32, NumberParseOption) /__w/1/s/test/Microsoft.ML.TestFramework/BaseTestBaseline.cs 409
[m[37m Boolean CheckEqualityNormalized(System.String, System.String, System.String, Int32, NumberParseOption) /__w/1/s/test/Microsoft.ML.TestFramework/BaseTestBaseline.cs 377
[m[37m Boolean CheckEqualityNormalized(Int32, NumberParseOption) /__w/1/s/test/Microsoft.ML.TestFramework/TestCommandBase.cs 79
[m[37m Void Run(RunContext, Int32, NumberParseOption) /__w/1/s/test/Microsoft.ML.TestFramework/BaseTestPredictorsMaml.cs 169
[m[37m Void Run_CV(Microsoft.ML.RunTests.PredictorAndArgs, Microsoft.ML.TestFrameworkCommon.TestDataset, System.String[], System.String, Boolean, Int32, NumberParseOption) /__w/1/s/test/Microsoft.ML.TestFramework/BaseTestPredictorsMaml.cs 442
[m[37m Void RunOneAllTests(Microsoft.ML.RunTests.PredictorAndArgs, Microsoft.ML.TestFrameworkCommon.TestDataset, System.String[], System.String, Boolean, Int32, NumberParseOption) /__w/1/s/test/Microsoft.ML.TestFramework/BaseTestPredictorsMaml.cs 363
[m[37m Void MulticlassLRTest() /__w/1/s/test/Microsoft.ML.Predictor.Tests/TestPredictors.cs 155
[m[37m System.Object InvokeMethod(System.Object, System.Span`1[System.Object] ByRef, System.Signature, Boolean, Boolean) 0
[m[37m System.Object Invoke(System.Object, System.Reflection.BindingFlags, System.Reflection.Binder, System.Object[], System.Globalization.CultureInfo) 0
[m[37m System.Object CallTestMethod(System.Object) /_/src/xunit.execution/Sdk/Frameworks/Runners/TestInvoker.cs 150
[m[37m Void MoveNext() /_/src/xunit.execution/Sdk/Frameworks/Runners/TestInvoker.cs 257
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m System.Threading.Tasks.Task b__1() 0
[m[37m Void MoveNext() /_/src/xunit.execution/Sdk/Frameworks/ExecutionTimer.cs 48
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m System.Threading.Tasks.Task AggregateAsync(System.Func`1[System.Threading.Tasks.Task]) 0
[m[37m System.Threading.Tasks.Task b__0() /_/src/xunit.execution/Sdk/Frameworks/Runners/TestInvoker.cs 242
[m[37m Void MoveNext() /_/src/xunit.core/Sdk/ExceptionAggregator.cs 90
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m System.Threading.Tasks.Task RunAsync(System.Func`1[System.Threading.Tasks.Task]) 0
[m[37m Void MoveNext() /_/src/xunit.execution/Sdk/Frameworks/Runners/TestInvoker.cs 241
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m System.Threading.Tasks.Task`1[System.Decimal] InvokeTestMethodAsync(System.Object) 0
[m[37m System.Threading.Tasks.Task`1[System.Decimal] InvokeTestMethodAsync(System.Object) /_/src/xunit.execution/Sdk/Frameworks/Runners/XunitTestInvoker.cs 112
[m[37m Void MoveNext() /_/src/xunit.execution/Sdk/Frameworks/Runners/TestInvoker.cs 206
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m System.Threading.Tasks.Task`1[System.Decimal] b__47_0() 0
[m[37m Void MoveNext() /_/src/xunit.core/Sdk/ExceptionAggregator.cs 107
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m System.Threading.Tasks.Task`1[T] RunAsync[T](System.Func`1[System.Threading.Tasks.Task`1[T]]) 0
[m[37m System.Threading.Tasks.Task`1[System.Decimal] RunAsync() /_/src/xunit.execution/Sdk/Frameworks/Runners/TestInvoker.cs 189
[m[37m System.Threading.Tasks.Task`1[System.Decimal] InvokeTestMethodAsync(Xunit.Sdk.ExceptionAggregator) /_/src/xunit.execution/Sdk/Frameworks/Runners/XunitTestRunner.cs 84
[m[37m Void MoveNext() /_/src/xunit.execution/Sdk/Frameworks/Runners/XunitTestRunner.cs 67
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m System.Threading.Tasks.Task`1[System.Tuple`2[System.Decimal,System.String]] InvokeTestAsync(Xunit.Sdk.ExceptionAggregator) 0
[m[37m System.Threading.Tasks.Task`1[System.Tuple`2[System.Decimal,System.String]] b__0() /_/src/xunit.execution/Sdk/Frameworks/Runners/TestRunner.cs 149
[m[37m Void MoveNext() /_/src/xunit.core/Sdk/ExceptionAggregator.cs 107
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m System.Threading.Tasks.Task`1[T] RunAsync[T](System.Func`1[System.Threading.Tasks.Task`1[T]]) 0
[m[37m Void MoveNext() 0
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m System.Threading.Tasks.Task`1[Xunit.Sdk.RunSummary] RunAsync() 0
[m[37m Void MoveNext() /_/src/xunit.execution/Sdk/Frameworks/Runners/TestCaseRunner.cs 82
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m System.Threading.Tasks.Task`1[Xunit.Sdk.RunSummary] RunAsync() 0
[m[37m Void MoveNext() /_/src/xunit.execution/Sdk/Frameworks/Runners/TestMethodRunner.cs 136
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m System.Threading.Tasks.Task`1[Xunit.Sdk.RunSummary] RunTestCasesAsync() 0
[m[37m Void MoveNext() /_/src/xunit.execution/Sdk/Frameworks/Runners/TestMethodRunner.cs 106
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m System.Threading.Tasks.Task`1[Xunit.Sdk.RunSummary] RunAsync() 0
[m[37m Void MoveNext() /_/src/xunit.execution/Sdk/Frameworks/Runners/TestClassRunner.cs 213
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m System.Threading.Tasks.Task`1[Xunit.Sdk.RunSummary] RunTestMethodsAsync() 0
[m[37m Void MoveNext() /_/src/xunit.execution/Sdk/Frameworks/Runners/TestClassRunner.cs 171
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m System.Threading.Tasks.Task`1[Xunit.Sdk.RunSummary] RunAsync() 0
[m[37m System.Threading.Tasks.Task`1[Xunit.Sdk.RunSummary] RunTestClassAsync(Xunit.Abstractions.ITestClass, Xunit.Abstractions.IReflectionTypeInfo, System.Collections.Generic.IEnumerable`1[Xunit.Sdk.IXunitTestCase]) /_/src/xunit.execution/Sdk/Frameworks/Runners/XunitTestCollectionRunner.cs 158
[m[37m Void MoveNext() /_/src/xunit.execution/Sdk/Frameworks/Runners/TestCollectionRunner.cs 130
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m System.Threading.Tasks.Task`1[Xunit.Sdk.RunSummary] RunTestClassesAsync() 0
[m[37m Void MoveNext() /_/src/xunit.execution/Sdk/Frameworks/Runners/TestCollectionRunner.cs 101
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m System.Threading.Tasks.Task`1[Xunit.Sdk.RunSummary] RunAsync() 0
[m[37m System.Threading.Tasks.Task`1[Xunit.Sdk.RunSummary] RunTestCollectionAsync(Xunit.Sdk.IMessageBus, Xunit.Abstractions.ITestCollection, System.Collections.Generic.IEnumerable`1[Xunit.Sdk.IXunitTestCase], System.Threading.CancellationTokenSource) /_/src/xunit.execution/Sdk/Frameworks/Runners/XunitTestAssemblyRunner.cs 235
[m[37m System.Threading.Tasks.Task`1[Xunit.Sdk.RunSummary] b__2() /_/src/xunit.execution/Sdk/Frameworks/Runners/XunitTestAssemblyRunner.cs 184
[m[37m Void InnerInvoke() 0
[m[37m Void <.cctor>b__272_0(System.Object) 0
[m[37m Void RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object) 0
[m[37m Void ExecuteWithThreadLocal(System.Threading.Tasks.Task ByRef, System.Threading.Thread) 0
[m[37m Boolean ExecuteEntry() 0
[m[37m Void <.cctor>b__8_0(System.Object) 0
[m[37m Void RunOnSyncContext(System.Threading.SendOrPostCallback, System.Object) /_/src/xunit.execution/Sdk/MaxConcurrencySyncContext.cs 106
[m[37m Void b__0(System.Object) /_/src/xunit.execution/Sdk/MaxConcurrencySyncContext.cs 96
[m[37m Void RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object) 0
[m[37m Void WorkerThreadProc() /_/src/xunit.execution/Sdk/MaxConcurrencySyncContext.cs 96
[m[37m Void b__5_0(System.Object) /_/src/common/XunitWorkerThread.cs 37
[m[37m Void InnerInvoke() 0
[m[37m Void <.cctor>b__272_0(System.Object) 0
[m[37m Void RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object) 0
[m[37m Void ExecuteWithThreadLocal(System.Threading.Tasks.Task ByRef, System.Threading.Thread) 0
[m[37m Void ExecuteEntryUnsafe(System.Threading.Thread) 0
[m[37m Void <.cctor>b__10_0(System.Object) 0
[m[37m Void StartCallback() 0
[m[37m *** Failure #2: Output file not found: /root/helix/work/workitem/e/TestOutput/MulticlassLogisticRegression/MulticlassLogisticRegression-CV-iris.txt
[m[37m Void Fail(System.String, System.Object[]) /__w/1/s/test/Microsoft.ML.TestFramework/BaseTestBaseline.cs 224
[m[37m Boolean CheckOutFile(System.String) /__w/1/s/test/Microsoft.ML.TestFramework/BaseTestBaseline.cs 922
[m[37m Boolean CheckEqualityCore(System.String, System.String, System.String, Boolean, Int32, NumberParseOption) /__w/1/s/test/Microsoft.ML.TestFramework/BaseTestBaseline.cs 398
[m[37m Boolean CheckEquality(System.String, System.String, System.String, Int32, NumberParseOption) /__w/1/s/test/Microsoft.ML.TestFramework/BaseTestBaseline.cs 367
[m[37m Boolean CheckEquality(Int32, NumberParseOption) /__w/1/s/test/Microsoft.ML.TestFramework/TestCommandBase.cs 73
[m[37m Void Run(RunContext, Int32, NumberParseOption) /__w/1/s/test/Microsoft.ML.TestFramework/BaseTestPredictorsMaml.cs 218
[m[37m Void Run_CV(Microsoft.ML.RunTests.PredictorAndArgs, Microsoft.ML.TestFrameworkCommon.TestDataset, System.String[], System.String, Boolean, Int32, NumberParseOption) /__w/1/s/test/Microsoft.ML.TestFramework/BaseTestPredictorsMaml.cs 442
[m[37m Void RunOneAllTests(Microsoft.ML.RunTests.PredictorAndArgs, Microsoft.ML.TestFrameworkCommon.TestDataset, System.String[], System.String, Boolean, Int32, NumberParseOption) /__w/1/s/test/Microsoft.ML.TestFramework/BaseTestPredictorsMaml.cs 363
[m[37m Void MulticlassLRTest() /__w/1/s/test/Microsoft.ML.Predictor.Tests/TestPredictors.cs 155
[m[37m System.Object InvokeMethod(System.Object, System.Span`1[System.Object] ByRef, System.Signature, Boolean, Boolean) 0
[m[37m System.Object Invoke(System.Object, System.Reflection.BindingFlags, System.Reflection.Binder, System.Object[], System.Globalization.CultureInfo) 0
[m[37m System.Object CallTestMethod(System.Object) /_/src/xunit.execution/Sdk/Frameworks/Runners/TestInvoker.cs 150
[m[37m Void MoveNext() /_/src/xunit.execution/Sdk/Frameworks/Runners/TestInvoker.cs 257
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m System.Threading.Tasks.Task b__1() 0
[m[37m Void MoveNext() /_/src/xunit.execution/Sdk/Frameworks/ExecutionTimer.cs 48
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m System.Threading.Tasks.Task AggregateAsync(System.Func`1[System.Threading.Tasks.Task]) 0
[m[37m System.Threading.Tasks.Task b__0() /_/src/xunit.execution/Sdk/Frameworks/Runners/TestInvoker.cs 242
[m[37m Void MoveNext() /_/src/xunit.core/Sdk/ExceptionAggregator.cs 90
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m System.Threading.Tasks.Task RunAsync(System.Func`1[System.Threading.Tasks.Task]) 0
[m[37m Void MoveNext() /_/src/xunit.execution/Sdk/Frameworks/Runners/TestInvoker.cs 241
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m System.Threading.Tasks.Task`1[System.Decimal] InvokeTestMethodAsync(System.Object) 0
[m[37m System.Threading.Tasks.Task`1[System.Decimal] InvokeTestMethodAsync(System.Object) /_/src/xunit.execution/Sdk/Frameworks/Runners/XunitTestInvoker.cs 112
[m[37m Void MoveNext() /_/src/xunit.execution/Sdk/Frameworks/Runners/TestInvoker.cs 206
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m System.Threading.Tasks.Task`1[System.Decimal] b__47_0() 0
[m[37m Void MoveNext() /_/src/xunit.core/Sdk/ExceptionAggregator.cs 107
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m System.Threading.Tasks.Task`1[T] RunAsync[T](System.Func`1[System.Threading.Tasks.Task`1[T]]) 0
[m[37m System.Threading.Tasks.Task`1[System.Decimal] RunAsync() /_/src/xunit.execution/Sdk/Frameworks/Runners/TestInvoker.cs 189
[m[37m System.Threading.Tasks.Task`1[System.Decimal] InvokeTestMethodAsync(Xunit.Sdk.ExceptionAggregator) /_/src/xunit.execution/Sdk/Frameworks/Runners/XunitTestRunner.cs 84
[m[37m Void MoveNext() /_/src/xunit.execution/Sdk/Frameworks/Runners/XunitTestRunner.cs 67
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m System.Threading.Tasks.Task`1[System.Tuple`2[System.Decimal,System.String]] InvokeTestAsync(Xunit.Sdk.ExceptionAggregator) 0
[m[37m System.Threading.Tasks.Task`1[System.Tuple`2[System.Decimal,System.String]] b__0() /_/src/xunit.execution/Sdk/Frameworks/Runners/TestRunner.cs 149
[m[37m Void MoveNext() /_/src/xunit.core/Sdk/ExceptionAggregator.cs 107
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m System.Threading.Tasks.Task`1[T] RunAsync[T](System.Func`1[System.Threading.Tasks.Task`1[T]]) 0
[m[37m Void MoveNext() 0
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m System.Threading.Tasks.Task`1[Xunit.Sdk.RunSummary] RunAsync() 0
[m[37m Void MoveNext() /_/src/xunit.execution/Sdk/Frameworks/Runners/TestCaseRunner.cs 82
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m System.Threading.Tasks.Task`1[Xunit.Sdk.RunSummary] RunAsync() 0
[m[37m Void MoveNext() /_/src/xunit.execution/Sdk/Frameworks/Runners/TestMethodRunner.cs 136
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m System.Threading.Tasks.Task`1[Xunit.Sdk.RunSummary] RunTestCasesAsync() 0
[m[37m Void MoveNext() /_/src/xunit.execution/Sdk/Frameworks/Runners/TestMethodRunner.cs 106
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m System.Threading.Tasks.Task`1[Xunit.Sdk.RunSummary] RunAsync() 0
[m[37m Void MoveNext() /_/src/xunit.execution/Sdk/Frameworks/Runners/TestClassRunner.cs 213
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m System.Threading.Tasks.Task`1[Xunit.Sdk.RunSummary] RunTestMethodsAsync() 0
[m[37m Void MoveNext() /_/src/xunit.execution/Sdk/Frameworks/Runners/TestClassRunner.cs 171
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m System.Threading.Tasks.Task`1[Xunit.Sdk.RunSummary] RunAsync() 0
[m[37m System.Threading.Tasks.Task`1[Xunit.Sdk.RunSummary] RunTestClassAsync(Xunit.Abstractions.ITestClass, Xunit.Abstractions.IReflectionTypeInfo, System.Collections.Generic.IEnumerable`1[Xunit.Sdk.IXunitTestCase]) /_/src/xunit.execution/Sdk/Frameworks/Runners/XunitTestCollectionRunner.cs 158
[m[37m Void MoveNext() /_/src/xunit.execution/Sdk/Frameworks/Runners/TestCollectionRunner.cs 130
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m System.Threading.Tasks.Task`1[Xunit.Sdk.RunSummary] RunTestClassesAsync() 0
[m[37m Void MoveNext() /_/src/xunit.execution/Sdk/Frameworks/Runners/TestCollectionRunner.cs 101
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m Void Start[TStateMachine](TStateMachine ByRef) 0
[m[37m System.Threading.Tasks.Task`1[Xunit.Sdk.RunSummary] RunAsync() 0
[m[37m System.Threading.Tasks.Task`1[Xunit.Sdk.RunSummary] RunTestCollectionAsync(Xunit.Sdk.IMessageBus, Xunit.Abstractions.ITestCollection, System.Collections.Generic.IEnumerable`1[Xunit.Sdk.IXunitTestCase], System.Threading.CancellationTokenSource) /_/src/xunit.execution/Sdk/Frameworks/Runners/XunitTestAssemblyRunner.cs 235
[m[37m System.Threading.Tasks.Task`1[Xunit.Sdk.RunSummary] b__2() /_/src/xunit.execution/Sdk/Frameworks/Runners/XunitTestAssemblyRunner.cs 184
[m[37m Void InnerInvoke() 0
[m[37m Void <.cctor>b__272_0(System.Object) 0
[m[37m Void RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object) 0
[m[37m Void ExecuteWithThreadLocal(System.Threading.Tasks.Task ByRef, System.Threading.Thread) 0
[m[37m Boolean ExecuteEntry() 0
[m[37m Void <.cctor>b__8_0(System.Object) 0
[m[37m Void RunOnSyncContext(System.Threading.SendOrPostCallback, System.Object) /_/src/xunit.execution/Sdk/MaxConcurrencySyncContext.cs 106
[m[37m Void b__0(System.Object) /_/src/xunit.execution/Sdk/MaxConcurrencySyncContext.cs 96
[m[37m Void RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object) 0
[m[37m Void WorkerThreadProc() /_/src/xunit.execution/Sdk/MaxConcurrencySyncContext.cs 96
[m[37m Void b__5_0(System.Object) /_/src/common/XunitWorkerThread.cs 37
[m[37m Void InnerInvoke() 0
[m[37m Void <.cctor>b__272_0(System.Object) 0
[m[37m Void RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object) 0
[m[37m Void ExecuteWithThreadLocal(System.Threading.Tasks.Task ByRef, System.Threading.Thread) 0
[m[37m Void ExecuteEntryUnsafe(System.Threading.Thread) 0
[m[37m Void <.cctor>b__10_0(System.Object) 0
[m[37m Void StartCallback() 0
[m[37m Test MulticlassLRTest: completed normally: failed
[m[37m Test MulticlassLRTest is using netcoreapp configuration specific baselines.
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

PR Validation Flaky Test

Description

Contributor guide

Assessment

Get new issues in your inbox