google / google/caliper

allocation intrument breaks my benchmark because it has non-deterministic allocations.

Open
#281 0 comments 0 reactions 0 assignees View on GitHub
component: runner P4 type=defect
Dominant language
Java
Stars
818
Forks
112
Avg merge
12m
Merged PRs (30d)
5

Description

```
I was attempting to benchmark some code that uses guice (specifically creates a
guice injector that tracks some multibinders) however I was unable to run the
allocation instrument because I was getting non-deterministic allocation
exceptions due to allocations that occurred from within the allocation
instrumenting code (we must go deeper).

here is a representative stack trace:
Experiment selection:
Instruments: [allocation]
User parameters: {graphSize=[MEDIUM], style=[DECLARATIVE]}
Virtual machines: [default]
Selection type: Full cartesian product

This selection yields 1 experiments.
Starting trial 1 of 1: {instrument=allocation, benchmarkMethod=createInjector,
vm=default, parameters={graphSize=MEDIUM, style=DECLARATIVE}}…
com.google.caliper.runner.ProxyWorkerException: An exception occurred in a
worker process. The stack trace is as follows:
java.lang.IllegalStateException: Your benchmark appears to have non-deterministic allocation behavior. Observed 1 instance(s) of char[10] (40 bytes)
at java.util.Arrays.copyOfRange(Arrays.java:2694)
at java.lang.String.(String.java:234)
at com.google.monitoring.runtime.instrumentation.asm.ClassReader.readUTF(ClassReader.java:2137)
at com.google.monitoring.runtime.instrumentation.asm.ClassReader.copyPool(ClassReader.java:327)
at com.google.monitoring.runtime.instrumentation.asm.ClassWriter.(ClassWriter.java:645)
at com.google.monitoring.runtime.instrumentation.StaticClassWriter.(StaticClassWriter.java:82)
at com.google.monitoring.runtime.instrumentation.AllocationInstrumenter.instrument(AllocationInstrumenter.java:168)
at com.google.monitoring.runtime.instrumentation.AllocationInstrumenter.instrument(AllocationInstrumenter.java:195)
at com.google.monitoring.runtime.instrumentation.AllocationInstrumenter.transform(AllocationInstrumenter.java:146)
at sun.instrument.TransformerManager.transform(TransformerManager.java:188)
at sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:424)
at java.util.ArrayList$Itr.(ArrayList.java:780)
at java.util.ArrayList.iterator(ArrayList.java:774)
at sun.reflect.generics.visitor.Reifier.visitClassTypeSignature(Reifier.java:108)
at sun.reflect.generics.tree.ClassTypeSignature.accept(ClassTypeSignature.java:49)
at sun.reflect.annotation.AnnotationParser.parseSig(AnnotationParser.java:390)
at sun.reflect.annotation.AnnotationParser.parseAnnotation(AnnotationParser.java:200)
at sun.reflect.annotation.AnnotationParser.parseAnnotations2(AnnotationParser.java:88)
at sun.reflect.annotation.AnnotationParser.parseAnnotations(AnnotationParser.java:70)
at java.lang.reflect.Method.declaredAnnotations(Method.java:714)
at java.lang.reflect.Method.getAnnotation(Method.java:700)
at com.google.inject.spi.InjectionPoint.getAtInject(InjectionPoint.java:478)
at com.google.inject.spi.InjectionPoint.getInjectionPoints(InjectionPoint.java:676)
at com.google.inject.spi.InjectionPoint.forInstanceMethodsAndFields(InjectionPoint.java:366)
at com.google.inject.spi.InjectionPoint.forInstanceMethodsAndFields(InjectionPoint.java:385)
at com.google.inject.internal.BindingBuilder.toInstance(BindingBuilder.java:84)
......

So the non-deterministic allocation is actually from within the
allocationinstrumenter! Either the instrumenter should be modified to not
record its own allocations or caliper should be modified to eliminate them,
since these allocations are actually being introduced by caliper, not my
benchmark!
```

Original issue reported on code.google.com by `lu...@google.com` on 9 Dec 2013 at 6:01

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.