Bench

A benchmark utility library for dart.

Build Status

Goals

  • Rapid definition of benchmark functions to cover libraries and APIs.
  • Simple and flexible configuration.
  • Automated harness to discover and run the benchmarks without manual wiring.

Quick Start

Write a Benchmark

Write a top-level function such as the following:

Benchmark affineTransformScale() {    
  var tx = new AffineTransform(1, 2, 3, 4, 5, 6);
  return new Benchmark(() => tx.scale(2, 3));
}

Here we have a top-level function named affineTransformScale that returns a value of type Benchmark. The Benchmark constructor requires only a method to invoke as the entry point to your algorithm:

Benchmark(void method(), {description:"", measure:100, warmup:200})

In the above example, the closure over tx.scale(2,3) is the method. Bench will invoke this method for the configured number of measure and warmup iterations, which have default values as shown above.

You write your setup routine in the lexical scope of your top-level function. In this example, the setup consists of constructing an instance of an AffineTransform for use in the benchmark:

var tx = new AffineTransform(1, 2, 3, 4, 5, 6);

We want to do this in our setup, so that the cost of constructing the transform does not play a part in the benchmark. This means our benchmark will only be measuring the performance of the scale method.

Run the Benchmarker

The library provides a Benchmarker class that you use to run the harness:

void main() {
  new Benchmarker().run();
}

The Benchmarker may be configured in a number of ways, but it can be used as simply as shown above.

Parse the Result

Every run of the Benchmarker will complete with a new BenchmarkResult object. The default behavior will use the logging library to log the result. For our affineTransformScale benchmark we'll get:

initializing isolate: affinity_bench.dart/main-7113
running benchmarks...
affinity_bench.affineTransformScale : (36 ms / 10000 iterations) = 0.0036

You may override this by providing a BenchmarkHandler to the handler argument of the run method:

void myFancyResultHandler(BenchmarkResult result) {
  // do something fancy!
}

new Benchmarker().run(handler:myFancyResultHandler);

Or, if you want to augment the default handler (rather than override it), the run method will complete the BenchmarkResult in the Future of its return value:

new Benchmarker().run().then((result) {
  // perform some custom parsing of the BenchmarkResult
}

Bench uses the MIT license as described in the LICENSE file, and follows semantic versioning.

Libraries

bench

A library for executing micro benchmark functions.