BenchmarkDotNet running same benchmark multiple times

Question

Faced with the issue that my current configuration for the benchmarks did not working properly, the reason of it is that I'm trying to run the benchmark only ones, for that as it mentioned in documentation I've to use this attribute [SimpleJob(RunStrategy.ColdStart, targetCount: 1)] and this was getting me in the wrong direction because from the console I've notice that my single bench was founded twice.

// ***** BenchmarkRunner: Start ***** // ***** Found 2 benchmark(s) in total ***** // ***** Building 2 exe(s) in Parallel: Start ***** internal class Program { static async Task Main(string[] args) => BenchmarkSwitcher.FromAssembly(typeof(Program).Assembly).Run(args, new DebugInProcessConfig()); } [MarkdownExporter] [AsciiDocExporter] [HtmlExporter] [CsvExporter] [RPlotExporter] [SimpleJob(RunStrategy.ColdStart, targetCount: 1)] public class MyBench { [Params(2)] public int _anotherValueToTestWith; [Params(2)] public int _valueToTestWith; [GlobalSetup] public void GlobalSetup() { // ... } [GlobalCleanup] public void GlobalCleanup() { // CleanUp } [Benchmark] public void AccessTokenServiceBench() { // Perform bench } }

What I'm missing here?

In the docs the iteration count is controlled by IterationCount, MaxIterationCount and MinIterationCount. There's no explicit definition for targetCount — Panagiotis Kanavos
– Panagiotis Kanavos, Commented Feb 18, 2022 at 12:41
@PanagiotisKanavos You are right, but, if you take a look to this example link, you will see at the Output section that it was iterated 5 times. Also tried as it mention for the IterationCount still same behavior — semper fi
– semper fi, Commented Feb 18, 2022 at 13:04
For some reason, if I would change the entry point of execution to this static void Main(string[] args) => BenchmarkRunner.Run<MyBench>(); It would run it properly, at once, but I would not be able to debug it. — semper fi
– semper fi, Commented Feb 18, 2022 at 13:12

Adam Sitnik · Accepted Answer · 2022-02-21 12:14:31Z

The DebugInProcessConfig defines one Job:

public override IEnumerable<Job> GetJobs() => new[] { Job.Default

and you add one more using the attribute:

[SimpleJob(RunStrategy.ColdStart, targetCount: 1)]

so the resulting config ends up having two jobs. And BDN runs every benchmark for every job defined.

The easiest fix would be to define your own config that runs the benchmark once (Job.Dry) with the desired toolchain:

public class DebugInProcessConfigDry : DebugConfig { public override IEnumerable<Job> GetJobs() => new[] { Job.Dry // Job.Dry instead of Job.Default .WithToolchain( new InProcessEmitToolchain( TimeSpan.FromHours(1), // 1h should be enough to debug the benchmark true)) }; }

Collectives™ on Stack Overflow

BenchmarkDotNet running same benchmark multiple times

1 Answer 1

Comments

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Related