dotnet / dotnet/BenchmarkDotNet

Isolate InProcess benchmarks

Open
#2,417 17 comments 0 reactions 0 assignees View on GitHub
Area:Toolchains enhancement
Dominant language
C#
Stars
11.5k
Forks
1.1k
Avg merge
6d 11h
Merged PRs (30d)
12

Description

One major downside of in-process benchmarks is that, if they produce side-effects, they can affect the results of other benchmarks. See #2197

Currently, in-process benchmarks run on their own dedicated thread to get a semblance of isolation, but it doesn't isolate global state.

I think we can obtain full isolation of the benchmarks using `AssemblyLoadContext` in Core, and `AppDomain` in Framework. We target `netstandard2.0` which only supports `AppDomain`, but of course it doesn't work in Core. So we'll have to use reflection to use `AssemblyLoadContext`, but I think it will be worth it. [Edit] Or it looks like we could use `System.Runtime.Loader` nuget package, or add `netcoreapp2.0` target.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.