`Sequence contains no elements` Exception in `Reporter.cs` in F# benchmark
- Dominant language
- F#
- Stars
- 773
- Forks
- 301
- Avg merge
- 6d 9h
- Merged PRs (30d)
- 15
Description
There's an exception thrown in [`tools/Reporting/Reporting/Reporter.cs`](https://github.com/dotnet/performance/blob/11a3e1e86c6c09fcf76289959dab0ea68fd615c7/src/tools/Reporting/Reporting/Reporter.cs#L129) that fails the scenario benchmark for [F# Compiler Service](https://github.com/dotnet/performance/tree/main/src/scenarios/fsharpcompilerservice). It seemed like an intermittent issue initially but now it seems to be happening every time - on **Ubuntu.1804.Amd64.Open**
The issues is that `Counter.Results` is empty and therefore LINQ aggregations (`Min/Max/Average`) throw `System.InvalidOperationException: Sequence contains no elements`
- [Example CI run](https://dev.azure.com/dnceng-public/public/_build/results?buildId=38304&view=ms.vss-test-web.build-test-results-tab&runId=764332&resultId=100069&paneView=dotnet-dnceng.dnceng-build-release-tasks.helix-test-information-tab)
- [Full log](https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-performance-refs-heads-main-0146089f16284ee5a5/F%23%20Compiler%20Service%20Clean%20Build/1/console.8ef13c12.log?helixlogtype=result)
Full stack trace:
```
[2022/10/03 12:20:50][INFO] Parsing traces/F#_Compiler_Service_Clean_Build_startup.trace.zip
[2022/10/03 12:20:50][INFO] Unhandled exception: System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
[2022/10/03 12:20:50][INFO] ---> System.InvalidOperationException: Sequence contains no elements
[2022/10/03 12:20:50][INFO] at System.Linq.ThrowHelper.ThrowNoElementsException()
[2022/10/03 12:20:50][INFO] at System.Linq.Enumerable.MaxFloat[T](IEnumerable`1 source)
[2022/10/03 12:20:50][INFO] at Reporting.Reporter.<>c.b__10_4(Counter c) in /__w/1/s/src/tools/Reporting/Reporting/Reporter.cs:line 129
[2022/10/03 12:20:50][INFO] at System.Linq.Enumerable.MaxInteger[TSource,TResult](IEnumerable`1 source, Func`2 selector)
[2022/10/03 12:20:50][INFO] at Reporting.Reporter.WriteResultTable() in /__w/1/s/src/tools/Reporting/Reporting/Reporter.cs:line 129
[2022/10/03 12:20:50][INFO] at ScenarioMeasurement.Startup.CreateTestReport(String scenarioName, IEnumerable`1 counters, String reportJsonPath, Logger logger) in /__w/1/s/src/tools/ScenarioMeasurement/Startup/Startup.cs:line 533
[2022/10/03 12:20:50][INFO] at ScenarioMeasurement.Startup.Main(String appExe, MetricType metricType, String scenarioName, String traceName, Boolean processWillExit, Int32 iterations, String iterationSetup, String setupArgs, String iterationCleanup, String cleanupArgs, Int32 timeout, Int32 measurementDelay, String appArgs, String logFileName, String workingDir, Boolean warmup, Boolean guiApp, Boolean skipProfileIteration, String reportJsonPath, String traceDirectory, String environmentVariables, String innerLoopCommand, String innerLoopCommandArgs, Boolean runWithoutExit, Int32 hotReloadIters, Boolean skipMeasurementIteration, Boolean parseOnly, Boolean runWithDotnet) in /__w/1/s/src/tools/ScenarioMeasurement/Startup/Startup.cs:line 337
[2022/10/03 12:20:50][INFO] at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
[2022/10/03 12:20:50][INFO] at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)
[2022/10/03 12:20:50][INFO] --- End of inner exception stack trace ---
[2022/10/03 12:20:50][INFO] at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)
[2022/10/03 12:20:50][INFO] at System.Reflection.RuntimeMethodInfo.InvokeWithManyArguments(RuntimeMethodInfo mi, Int32 argCount, Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
[2022/10/03 12:20:50][INFO] at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
[2022/10/03 12:20:50][INFO] at System.CommandLine.Invocation.ModelBindingCommandHandler.InvokeAsync(InvocationContext context)
[2022/10/03 12:20:50][INFO] at System.CommandLine.Invocation.InvocationPipeline.<>c__DisplayClass4_0.<b__0>d.MoveNext()
[2022/10/03 12:20:50][INFO] --- End of stack trace from previous location ---
[2022/10/03 12:20:50][INFO] at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c.<b__19_0>d.MoveNext()
[2022/10/03 12:20:50][INFO] --- End of stack trace from previous location ---
[2022/10/03 12:20:50][INFO] at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass14_0.<b__0>d.MoveNext()
[2022/10/03 12:20:50][INFO] --- End of stack trace from previous location ---
[2022/10/03 12:20:50][INFO] at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass22_0.<b__0>d.MoveNext()
[2022/10/03 12:20:50][INFO] --- End of stack trace from previous location ---
[2022/10/03 12:20:50][INFO] at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass21_0.<b__0>d.MoveNext()
[2022/10/03 12:20:50][INFO] --- End of stack trace from previous location ---
[2022/10/03 12:20:50][INFO] at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c.<b__20_0>d.MoveNext()
[2022/10/03 12:20:50][INFO] --- End of stack trace from previous location ---
[2022/10/03 12:20:50][INFO] at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c.<b__18_0>d.MoveNext()
[2022/10/03 12:20:50][INFO] --- End of stack trace from previous location ---
[2022/10/03 12:20:50][INFO] at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c.<b__10_0>d.MoveNext()
[2022/10/03 12:20:50][INFO] --- End of stack trace from previous location ---
[2022/10/03 12:20:50][INFO] at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c.<b__9_0>d.MoveNext()
[2022/10/03 12:20:50][INFO] --- End of stack trace from previous location ---
[2022/10/03 12:20:50][INFO] at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass5_0.<b__0>d.MoveNext()
[2022/10/03 12:20:50][INFO] --- End of stack trace from previous location ---
[2022/10/03 12:20:50][INFO] at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass12_0.<b__0>d.MoveNext()
[2022/10/03 12:20:50][ERROR] Process exited with status 1
[2022/10/03 12:20:50][INFO] Run failure registered
Traceback (most recent call last):
File "test.py", line 14, in
Runner(traits).run()
File "/datadisks/disk1/work/9D9C08CB/p/shared/runner.py", line 244, in run
startup.runtests(self.traits)
File "/datadisks/disk1/work/9D9C08CB/p/shared/startup.py", line 152, in runtests
RunCommand(startup_args, verbose=True).run()
File "/datadisks/disk1/work/9D9C08CB/p/scripts/performance/common.py", line 248, in run
returncode, quoted_cmdline)
subprocess.CalledProcessError: Command '$ /datadisks/disk1/work/9D9C08CB/p/startup/Startup --app-exe dotnet --metric-type ProcessTime --trace-name "F# Compiler Service Clean Build_startup" --gui-app false --process-will-exit true --iterations 1 --timeout 600 --warmup true --working-dir app --report-json-path traces/perf-lab-report.json --trace-directory traces --scenario-name "F# Compiler Service Clean Build" --app-args build --environment-variables MSBUILDDISABLENODEREUSE=1;DOTNET_MULTILEVEL_LOOKUP=0 --iteration-setup python3 --setup-args " /datadisks/disk1/work/9D9C08CB/p/shared/sdk_iteration_setup.py setup_build" --iteration-cleanup python3 --cleanup-args " /datadisks/disk1/work/9D9C08CB/p/shared/sdk_iteration_setup.py cleanup"' returned non-zero exit status 1.
```
Is this something that can be fixed in the F# scenario? If so, how?
Otherwise here's an example fix that would ignore empty results:
- https://github.com/0101/performance/commit/6a10733d50da0e8d3a223b7924e98fd9d0d46709 + https://github.com/0101/performance/commit/1e8a12263b4302252004646e4d1bfffc614a6c36
If we don't want to ignore empty results it might be worth it to throw a custom exception that would explain what's happening and how to fix it.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.