google / google/caliper

No timeout in dryRun()?

Open
#261 0 comments 0 reactions 0 assignees View on GitHub
component: api P4 type=addition
Dominant language
Java
Stars
818
Forks
112
Avg merge
12m
Merged PRs (30d)
5

Description

```
What steps will reproduce the problem?
1. Write a slow macro benchmark (e.g. large size, slow algorithm)
2. Run caliper
3. Caliper appears to freeze after "This selection yields %s experiments."

What is the expected output? What do you see instead?

I'd expect caliper to apply timeouts to the dryRun invocations, too; or give a
chance to skip them.
Performing a dry run is cheap for microbenchmarks, but for macrobenchmarks its
can be rather expensive. It would be nice, to be able to use the time limit -l
to skip benchmarks that are too slow.

In my particular case, I'm benchmarking different sorting implementations out
of curiosity. And of course I do have insertion sort and bubble sort in there,
too; but these scale really bad. My overall goal is to estimate real (average)
complexity for such implementations, so I wanted to have some measurements for
large n in there, too, to see if that improves my curve fitting.
```

Original issue reported on code.google.com by `erich.sc...@gmail.com` on 7 Jul 2013 at 11:01

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.