dotnet / dotnet/BenchmarkDotNet

ArgumentsSource method is executed before GlobalSetup method

Open
#1,408 4 comments 7 reactions 0 assignees View on GitHub
Area:Engine
Dominant language
C#
Stars
11.5k
Forks
1.1k
Avg merge
4d 11h
Merged PRs (30d)
11

Description

In **BenchmarkDotNet** version 0.12, the method that provided arguments for a parameterized benchmark is executed before the **GlobalSetup** method of the benchmark.

Having the designated **GlobalSetup** method execute before the **ArgumentsSource** method is useful when the arguments are costly to create, and - for example - need to be disposed once the benchmark has completed.

**Example:**
```cs
using BenchmarkDotNet.Attributes;
using System;
using System.Collections.Generic;
using System.IO;

public class ArgumentsSourceBenchmarks
{
private List _downloadFiles;
private StreamProvider _streamProvider;

[GlobalSetup]
public void GlobalSetup()
{
_streamProvider = new StreamProvider();
_downloadFiles = new List
{
new DownloadFile("A", _streamProvider.Create(2048)),
new DownloadFile("B", _streamProvider.Create(4096))
};
}

[GlobalCleanup]
public void GlobalCleanup()
{
_streamProvider?.Dispose();

foreach (var downloadFile in _downloadFiles)
{
downloadFile.DownloadStream.Dispose();
}
}

[Benchmark]
[ArgumentsSource(nameof(DownloadFileArguments))]
public void DownloadFileBenchmark(string path, Stream stream)
{
// Do something
}

public IEnumerable DownloadFileArguments()
{
foreach (var downloadFile in _downloadFiles)
{
yield return new object[] { downloadFile.Path, downloadFile.DownloadStream };
}
}

public class DownloadFile
{
public DownloadFile(string path, Stream downloadStream)
{
Path = path;
DownloadStream = downloadStream;
}

public string Path { get; }
public Stream DownloadStream { get; }
}

public class StreamProvider : IDisposable
{
public void Dispose()
{
}

public Stream Create(int initialCapacity)
{
return new MemoryStream(initialCapacity);
}
}
}
```

Running this benchmark class will current result in an NRE:

```txt
Unhandled Exception: System.NullReferenceException: Object reference not set to an instance of an object.
at ArgumentsSourceBenchmarks.d__4.MoveNext() in D:\Development\Repos\ArgumentsSource\ArgumentsSourceBenchmarks.cs:line 32
at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
at BenchmarkDotNet.Running.BenchmarkConverter.GetValidValuesForParamsSource(Type parentType, String sourceName)
at BenchmarkDotNet.Running.BenchmarkConverter.d__9.MoveNext()
at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
at BenchmarkDotNet.Running.BenchmarkConverter.MethodsToBenchmarksWithFullConfig(Type containingType, MethodInfo[] benchmarkMethods, ImmutableConfig immutableConfig)
at System.Linq.Enumerable.WhereSelectListIterator`2.MoveNext()
at System.Linq.Enumerable.WhereEnumerableIterator`1.MoveNext()
at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
at BenchmarkDotNet.Running.BenchmarkSwitcher.RunWithDirtyAssemblyResolveHelper(String[] args, IConfig config)
at BenchmarkDotNet.Running.BenchmarkSwitcher.Run(String[] args, IConfig config)
at Program.Main(String[] args) in D:\Development\Repos\ArgumentsSource\Program.cs:line 13
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.