bazelbuild / bazelbuild/rules_android
Conflicting version of R8/D8 in Dexer/Desugar classpath
- Dominant language
- Java
- Stars
- 203
- Forks
- 95
- PR merge metrics
- No merged PRs in 30d
Description
### Description of the bug:
We started seeing the following errors during dex merging when using D8
```
23:01:25 [Bazel] Error in bazel-out/darwin_x86_64-fastbuild/bin/apps/hermosa/matador-services/settings/dexsplits/hermosa-settings-service-debug/9.shard.zip:kotlinx/coroutines/rx3/RxSingleKt$$ExternalSyntheticLambda0.class.dex:
23:01:25 [Bazel] Attempt at compiling intermediate artifact without its context
23:01:25 [Bazel] Merge failed: Compilation failed to complete, origin: bazel-out/darwin_x86_64-fastbuild/bin/apps/hermosa/matador-services/settings/dexsplits/hermosa-settings-service-debug/9.shard.zip:kotlinx/coroutines/rx3/RxSingleKt$$ExternalSyntheticLambda0.class.dex
23:01:25 [Bazel] Exception in thread "main" java.lang.RuntimeException: Merge failed: Compilation failed to complete, origin: bazel-out/darwin_x86_64-fastbuild/bin/apps/hermosa/matador-services/settings/dexsplits/hermosa-settings-service-debug/9.shard.zip:kotlinx/coroutines/rx3/RxSingleKt$$ExternalSyntheticLambda0.class.dex
23:01:25 [Bazel] at com.google.devtools.build.android.r8.DexFileMerger.main(DexFileMerger.java:412)
23:01:25 [Bazel] Caused by: com.android.tools.r8.CompilationFailedException: Compilation failed to complete, origin: bazel-out/darwin_x86_64-fastbuild/bin/apps/hermosa/matador-services/settings/dexsplits/hermosa-settings-service-debug/9.shard.zip:kotlinx/coroutines/rx3/RxSingleKt$$ExternalSyntheticLambda0.class.dex
23:01:25 [Bazel] at Version.fakeStackEntry(Version_8.1.56.java:0)
23:01:25 [Bazel] at com.android.tools.r8.M.a(R8_8.1.56_756d1f50f618dd1c39c000f11defb367a21e9e866e3401b884be16c0950f6f79:5)
23:01:25 [Bazel] at com.android.tools.r8.utils.R0.a(R8_8.1.56_756d1f50f618dd1c39c000f11defb367a21e9e866e3401b884be16c0950f6f79:81)
23:01:25 [Bazel] at com.android.tools.r8.utils.R0.a(R8_8.1.56_756d1f50f618dd1c39c000f11defb367a21e9e866e3401b884be16c0950f6f79:32)
23:01:25 [Bazel] at com.android.tools.r8.utils.R0.a(R8_8.1.56_756d1f50f618dd1c39c000f11defb367a21e9e866e3401b884be16c0950f6f79:31)
23:01:25 [Bazel] at com.android.tools.r8.utils.R0.b(R8_8.1.56_756d1f50f618dd1c39c000f11defb367a21e9e866e3401b884be16c0950f6f79:2)
23:01:25 [Bazel] at com.android.tools.r8.D8.run(R8_8.1.56_756d1f50f618dd1c39c000f11defb367a21e9e866e3401b884be16c0950f6f79:6)
23:01:25 [Bazel] at com.google.devtools.build.android.r8.DexFileMerger.run(DexFileMerger.java:395)
23:01:25 [Bazel] at com.google.devtools.build.android.r8.DexFileMerger.main(DexFileMerger.java:409)
23:01:25 [Bazel] Caused by: com.android.tools.r8.utils.b: Attempt at compiling intermediate artifact without its context
23:01:25 [Bazel] at com.android.tools.r8.utils.O2.a(R8_8.1.56_756d1f50f618dd1c39c000f11defb367a21e9e866e3401b884be16c0950f6f79:21)
23:01:25 [Bazel] at com.android.tools.r8.utils.O2.error(R8_8.1.56_756d1f50f618dd1c39c000f11defb367a21e9e866e3401b884be16c0950f6f79:1)
23:01:25 [Bazel] at com.android.tools.r8.synthesis.D.a(R8_8.1.56_756d1f50f618dd1c39c000f11defb367a21e9e866e3401b884be16c0950f6f79:59)
23:01:25 [Bazel] at com.android.tools.r8.D8.d(R8_8.1.56_756d1f50f618dd1c39c000f11defb367a21e9e866e3401b884be16c0950f6f79:26)
23:01:25 [Bazel] at com.android.tools.r8.D8.a(R8_8.1.56_756d1f50f618dd1c39c000f11defb367a21e9e866e3401b884be16c0950f6f79:20)
23:01:25 [Bazel] at com.android.tools.r8.utils.R0.a(R8_8.1.56_756d1f50f618dd1c39c000f11defb367a21e9e866e3401b884be16c0950f6f79:28)
23:01:25 [Bazel] ... 4 more
23:01:25 [Bazel] Suppressed: java.lang.RuntimeException: com.android.tools.r8.utils.b: Attempt at compiling intermediate artifact without its context
23:01:25 [Bazel] at com.android.tools.r8.utils.O2.a(R8_8.1.56_756d1f50f618dd1c39c000f11defb367a21e9e866e3401b884be16c0950f6f79:29)
23:01:25 [Bazel] at com.android.tools.r8.dex.k.a(R8_8.1.56_756d1f50f618dd1c39c000f11defb367a21e9e866e3401b884be16c0950f6f79:225)
23:01:25 [Bazel] at com.android.tools.r8.D8.d(R8_8.1.56_756d1f50f618dd1c39c000f11defb367a21e9e866e3401b884be16c0950f6f79:149)
23:01:25 [Bazel] ... 6 more
23:01:25 [Bazel] Caused by: [CIRCULAR REFERENCE: com.android.tools.r8.utils.b: Attempt at compiling intermediate artifact without its context]
```
I did some investigation and the `META-INF/synthetic-contexts.map` in the `.jar.dex.zip` is incomplete. It looks under some circumstances an older version of D8 is being used during dexing. The version of R8 being used, is an older version which does not output information about synthetic classes and their contexts. The version that should be used should be the one pointed by [android_gmaven_r8](https://github.com/bazelbuild/bazel/blob/2b8219042c132483e0af39ef20d67dfd6442af01/distdir_deps.bzl#L332), but instead is the one bundled by the `@maven_android//:com_android_tools_build_builder_file` dep which was added as dependency in this commit https://github.com/bazelbuild/bazel/commit/aaf94c63bf1eb7db4921330e3f4eb0ede8381ec0#diff-bdb5a620f62ba1e7228456cde29d508737679c9196143db3641b22bb3ca2fd2aR128. If `com_android_tools_build_builde` happens to appear before than the R8 version coming from `android_gmave_r8` dep in the dexer classpath, then dexmerging is likely to fail due to missing synthetic classes info.
The [r8 target](https://github.com/bazelbuild/bazel/blob/2b8219042c132483e0af39ef20d67dfd6442af01/src/tools/android/java/com/google/devtools/build/android/r8/BUILD.tools#L8) shipped with the android tools depends on the gradle builder jar via `//src/tools/android/java/com/google/devtools/build/android:all_android_tools` which in turn transitively depends on the gradle jar. The gradle dependency needs to be teased out in order to avoid causing classpath conflicts.
The version used seems to be non-deterministic and depends on which commit we are building our app . Furthermore we've only encountered this bug in our linux builds and we have not been able to repro on Mac builds.
### Which category does this issue belong to?
Android
### What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.
This bug is not deterministic and we are only able to repro at specific commits when building our app, but its easy to check the bundled version of R8 in the [gradle jar](https://mvnrepository.com/artifact/com.android.tools.build/builder/7.4.2). Furthermore we've only encountered this bug in our linux builds and we have not been able to repro on Mac builds.
### Which operating system are you running Bazel on?
Linux
### What is the output of `bazel info release`?
release 7.0.2.13 (internal release)
### If `bazel info release` returns `development version` or `(@non-git)`, tell us how you built Bazel.
Built from internal forked version of bazel. Forked at this commit from upstream https://github.com/bazelbuild/bazel/commit/252d36384b8b630d77d21fac0d2c5608632aa393
### What's the output of `git remote get-url origin; git rev-parse master; git rev-parse HEAD` ?
_No response_
### Is this a regression? If yes, please try to identify the Bazel commit where the bug was introduced.
https://github.com/bazelbuild/bazel/commit/aaf94c63bf1eb7db4921330e3f4eb0ede8381ec0
### Have you found anything relevant by searching the web?
_No response_
### Any other information, logs, or outputs that you want to share?
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.