caliper measure pattern should support network bandwidth benchmarks
- Dominant language
- Java
- Stars
- 818
- Forks
- 112
- Avg merge
- 12m
- Merged PRs (30d)
- 5
Description
```
thank your for the caliper - great idea.
I have tried to use caliper for network bandwidth benchmarks with netty
https://github.com/netty/netty
in my experience current design pattern of caliper is unusable for this due to:
1) fixed reps invocation pattern:
does not suit well for non-blocking NIO paradigm;
2) only 3 dimensions can be reported/analysed:
run time, instance count, memory heap size;
re: 1) I suggest: instead of timing each run with fixed reps invocations,
have a fixed run time, and do run performance parameter sampling during the run,
like metrics does http://metrics.codahale.com/
re: 2) I suggest: allow for more measures and make those measures more
generic/configurable.
here is example how I hacked caliper v 0.5 to resolve both 1) and 2):
custom caliper runner
https://github.com/barchart/netty-udt/blob/master/transport-udt/src/test/java/io
/netty/transport/udt/util/CaliperRunner.java
custom caliper benchmark
https://github.com/barchart/netty-udt/blob/master/transport-udt/src/test/java/io
/netty/transport/udt/util/CaliperBench.java
custom caliper + metrics measure
https://github.com/barchart/netty-udt/blob/master/transport-udt/src/test/java/io
/netty/transport/udt/util/CaliperMeasure.java
cheers.
```
Original issue reported on code.google.com by `Andrei.Pozolotin` on 7 Jan 2013 at 5:17
Contributor guide
Assessment
This issue has not been assessed yet.