google / google/caliper

caliper measure pattern should support network bandwidth benchmarks

Open
#186 0 comments 0 reactions 0 assignees View on GitHub
component: runner P4 type=enhancement
Dominant language
Java
Stars
818
Forks
112
Avg merge
12m
Merged PRs (30d)
5

Description

```
thank your for the caliper - great idea.

I have tried to use caliper for network bandwidth benchmarks with netty
https://github.com/netty/netty

in my experience current design pattern of caliper is unusable for this due to:

1) fixed reps invocation pattern:
does not suit well for non-blocking NIO paradigm;

2) only 3 dimensions can be reported/analysed:
run time, instance count, memory heap size;

re: 1) I suggest: instead of timing each run with fixed reps invocations,
have a fixed run time, and do run performance parameter sampling during the run,
like metrics does http://metrics.codahale.com/

re: 2) I suggest: allow for more measures and make those measures more
generic/configurable.

here is example how I hacked caliper v 0.5 to resolve both 1) and 2):

custom caliper runner
https://github.com/barchart/netty-udt/blob/master/transport-udt/src/test/java/io
/netty/transport/udt/util/CaliperRunner.java

custom caliper benchmark
https://github.com/barchart/netty-udt/blob/master/transport-udt/src/test/java/io
/netty/transport/udt/util/CaliperBench.java

custom caliper + metrics measure
https://github.com/barchart/netty-udt/blob/master/transport-udt/src/test/java/io
/netty/transport/udt/util/CaliperMeasure.java

cheers.

```

Original issue reported on code.google.com by `Andrei.Pozolotin` on 7 Jan 2013 at 5:17

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.