Benchmark validation doesn't appear to be working on non-Windows OS
- Dominant language
- C#
- Stars
- 38.4k
- Forks
- 10.9k
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 281
Description
### Is there an existing issue for this?
- [X] I have searched the existing issues
### Describe the bug
Benchmarks written with benchmarkdotnet are validated on build:
https://github.com/dotnet/aspnetcore/blob/f0af11263e006d0df75b32d20d85bde5870a8d9c/eng/targets/CSharp.Common.targets#L133-L142
It appears that failures are only correctly reported on Windows. This behavior was noticed when I added a benchmark that is only valid for Windows, but tests all passed.
For example, I added an exception to a benchmark's setup here - https://github.com/dotnet/aspnetcore/pull/46491 - and only Windows tests failed:

I can see in the logs that they are being detected and run on other OS, but no error reported:

### Expected Behavior
Validation reports problems and the build fails on Linux and macOS.
### Steps To Reproduce
https://github.com/dotnet/aspnetcore/pull/46491
### Exceptions (if any)
_No response_
### .NET Version
_No response_
### Anything else?
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.