libfuzzer-coverage analyzes unrelated build plugin classes
- Dominant language
- Shell
- Stars
- 12.6k
- Forks
- 2.9k
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 62
Description
The Caffeine fuzz test [failed](https://oss-fuzz-build-logs.storage.googleapis.com/log-328039a0-bd87-4c47-a0ae-c82be4128855.txt) because the jacoco code coverage tool is used to analyze all *.class files regardless of their relevance to the fuzz execution. As the build tool compiles its plugins into class files, those are being picked up for analysis despite being unrelated to the project's own source code. The project uses Java 11, the build tool uses Java 21, and oss-fuzz infrastructure uses Java 17 for running its own tooling. This results in an unsupported class file version, even though each of these should be fully independent.
A best practice is to have Gradle builds organized as custom plugins that configure external plugins, such as testing or static analysis configuration. This way the project build files are short and declarative. This is known as [pre-compiled script plugins](https://docs.gradle.org/current/userguide/implementing_gradle_plugins_precompiled.html) where the plugins are their own project being built, the jar is placed onto the build tool's classpath, and then available when running the main project's build. This separation allows the build tool to use a different version of Java, which it can self-install, and it prefers the latest for the best developer experience. As a result, the generated class files are a mix of Java versions.
The oss-fuzz execution fails because it tries to parse the class file
```
./gradle/plugins/build/classes/kotlin/main/Spotbugs_caffeine_gradle$5.class
```
which is in Java 21 and completely unrelated to the library's code, e.g.
```
./caffeine/build/classes/java/main/com/github/benmanes/caffeine/cache/LoadingCache.class
```
The `libfuzzer-coverage` tool is run by the infrastructure, independent to the configuration provided in [projects/caffeine](https://github.com/google/oss-fuzz/tree/master/projects/caffeine). This means there is no way for a maintainer to resolve this problem directly. Instead, there is an implicit dependency that the covered projects must limit themselves to only use your Java version in their build infrastructure.
Would it be reasonable for the analyzer to treat this as a warning instead of error, skipping over the class files it cannot parse?
logs
```console
Step #5: Running CaffeineSpecFuzzer
Step #5: [INFO] Loading execution data file /workspace/out/libfuzzer-coverage-x86_64/dumps/CaffeineSpecFuzzer.exec.
Step #5: [INFO] Analyzing 65 classes.
Step #5: [INFO] Loading execution data file /workspace/out/libfuzzer-coverage-x86_64/dumps/CaffeineSpecFuzzer.exec.
Step #5: [INFO] Writing execution data to /workspace/out/libfuzzer-coverage-x86_64/dumps/jacoco.merged.exec.
Step #5: [INFO] Loading execution data file /workspace/out/libfuzzer-coverage-x86_64/dumps/jacoco.merged.exec.
Step #5: Exception in thread "main" java.io.IOException: Error while analyzing /workspace/out/libfuzzer-coverage-x86_64/dumps/classes/Spotbugs_caffeine_gradle$5.class.
Step #5: at org.jacoco.cli.internal.core.analysis.Analyzer.analyzerError(Analyzer.java:162)
Step #5: at org.jacoco.cli.internal.core.analysis.Analyzer.analyzeClass(Analyzer.java:134)
Step #5: at org.jacoco.cli.internal.core.analysis.Analyzer.analyzeClass(Analyzer.java:157)
Step #5: at org.jacoco.cli.internal.core.analysis.Analyzer.analyzeAll(Analyzer.java:193)
Step #5: at org.jacoco.cli.internal.core.analysis.Analyzer.analyzeAll(Analyzer.java:226)
Step #5: at org.jacoco.cli.internal.core.analysis.Analyzer.analyzeAll(Analyzer.java:221)
Step #5: at org.jacoco.cli.internal.commands.Report.analyze(Report.java:110)
Step #5: at org.jacoco.cli.internal.commands.Report.execute(Report.java:84)
Step #5: at org.jacoco.cli.internal.Main.execute(Main.java:90)
Step #5: at org.jacoco.cli.internal.Main.main(Main.java:105)
Step #5: Caused by: java.lang.IllegalArgumentException: Unsupported class file major version 65
Step #5: at org.jacoco.cli.internal.asm.ClassReader.(ClassReader.java:196)
Step #5: at org.jacoco.cli.internal.asm.ClassReader.(ClassReader.java:177)
Step #5: at org.jacoco.cli.internal.asm.ClassReader.(ClassReader.java:163)
Step #5: at org.jacoco.cli.internal.core.internal.instr.InstrSupport.classReaderFor(InstrSupport.java:280)
Step #5: at org.jacoco.cli.internal.core.analysis.Analyzer.analyzeClass(Analyzer.java:107)
Step #5: at org.jacoco.cli.internal.core.analysis.Analyzer.analyzeClass(Analyzer.java:132)
Step #5: ... 8 more
```
Contributor guide
Research direction
Start with the libfuzzer-coverage report path shown in the stack trace, especially the JaCoCo Report and Analyzer calls, and reproduce the failure using the linked Caffeine log. Done means the analyzer handles the unrelated Spotbugs_caffeine_gradle$5.class file without aborting, while still producing coverage output and reporting the skipped class appropriately.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- build-system, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100