google / google/ksp

Build stuck on kspDebugKotlin when using KSP on large project (dagger ksp)

Open
#1,720 9 comments 5 reactions 1 assignee Claimed by @ting-yuan View on GitHub
Dominant language
Kotlin
Stars
3.5k
Forks
415
Avg merge
1d 2h
Merged PRs (30d)
53

Description

We recently moved to ksp from kapt for dagger and we're facing some build getting stuck. Gradle is stuck on `kspDebugKotlin` on random module using dagger ksp.

We're using :
- Android Studio Jellyfish canary 8
- gradle 8.6
- android gradle plugin = 8.2.1
- kotlin = 1.9.22
- ksp = 1.9.22-1.0.17
- dagger = 2.49

We have ~300 modules.

I was able to get some thread stacks using YourKit :
```
RMI TCP Connection(558)-127.0.0.1 Runnable CPU usage on sample: 1s 2ms
java.io.BufferedInputStream.read1(BufferedInputStream.java:282)
java.io.BufferedInputStream.read(BufferedInputStream.java:343)
java.io.DataInputStream.readFully(DataInputStream.java:201)
java.io.DataInputStream.readFully(DataInputStream.java:172)
org.jetbrains.kotlin.com.intellij.util.io.IOUtil.readString(IOUtil.java:41)
com.google.devtools.ksp.FileKeyDescriptor.read(PersistentMap.kt:39)
com.google.devtools.ksp.FileKeyDescriptor.read(PersistentMap.kt:37)
org.jetbrains.kotlin.com.intellij.util.io.keyStorage.AppendableStorageBackedByResizableMappedFile.read(AppendableStorageBackedByResizableMappedFile.java:94)
org.jetbrains.kotlin.com.intellij.util.io.PersistentEnumeratorBase.findValueFor(PersistentEnumeratorBase.java:438)
org.jetbrains.kotlin.com.intellij.util.io.PersistentEnumeratorBase.lambda$valueOf$2(PersistentEnumeratorBase.java:430)
org.jetbrains.kotlin.com.intellij.util.io.PersistentEnumeratorBase$$Lambda$617.0x000000080192b290.compute()
org.jetbrains.kotlin.com.intellij.util.io.PersistentEnumeratorBase.catchCorruption(PersistentEnumeratorBase.java:608)
org.jetbrains.kotlin.com.intellij.util.io.PersistentEnumeratorBase.valueOf(PersistentEnumeratorBase.java:429)
org.jetbrains.kotlin.com.intellij.util.io.PersistentBTreeEnumerator.valueOf(PersistentBTreeEnumerator.java:667)
org.jetbrains.kotlin.com.intellij.util.io.PersistentEnumeratorBase.isKeyAtIndex(PersistentEnumeratorBase.java:373)
org.jetbrains.kotlin.com.intellij.util.io.PersistentBTreeEnumerator.enumerateImpl(PersistentBTreeEnumerator.java:528)
org.jetbrains.kotlin.com.intellij.util.io.PersistentEnumeratorBase.lambda$doEnumerate$0(PersistentEnumeratorBase.java:267)
org.jetbrains.kotlin.com.intellij.util.io.PersistentEnumeratorBase$$Lambda$572.0x00000008018a3008.compute()
org.jetbrains.kotlin.com.intellij.util.io.PersistentEnumeratorBase.catchCorruption(PersistentEnumeratorBase.java:608)
org.jetbrains.kotlin.com.intellij.util.io.PersistentEnumeratorBase.doEnumerate(PersistentEnumeratorBase.java:266)
org.jetbrains.kotlin.com.intellij.util.io.PersistentEnumeratorBase.enumerate(PersistentEnumeratorBase.java:279)
org.jetbrains.kotlin.com.intellij.util.io.PersistentMapImpl.enumerate(PersistentMapImpl.java:425)
org.jetbrains.kotlin.com.intellij.util.io.PersistentMapImpl.doPut(PersistentMapImpl.java:398)
org.jetbrains.kotlin.com.intellij.util.io.PersistentMapImpl.put(PersistentMapImpl.java:360)
org.jetbrains.kotlin.com.intellij.util.io.PersistentHashMap.put(PersistentHashMap.java:102)
com.google.devtools.ksp.PersistentMap.set(PersistentMap.kt:26)
com.google.devtools.ksp.IncrementalContextBase.updateCaches$update(IncrementalContextBase.kt:371)
com.google.devtools.ksp.IncrementalContextBase.updateCaches(IncrementalContextBase.kt:390)
com.google.devtools.ksp.IncrementalContextBase.updateCachesAndOutputs(IncrementalContextBase.kt:473)
com.google.devtools.ksp.AbstractKotlinSymbolProcessingExtension.doAnalysis(KotlinSymbolProcessingExtension.kt:358)
org.jetbrains.kotlin.cli.jvm.compiler.TopDownAnalyzerFacadeForJVM.analyzeFilesWithJavaIntegration(TopDownAnalyzerFacadeForJVM.kt:112)
org.jetbrains.kotlin.cli.jvm.compiler.TopDownAnalyzerFacadeForJVM.analyzeFilesWithJavaIntegration$default(TopDownAnalyzerFacadeForJVM.kt:77)
org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler$analyze$1.invoke(KotlinToJVMBytecodeCompiler.kt:256)
org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler$analyze$1.invoke(KotlinToJVMBytecodeCompiler.kt:247)
org.jetbrains.kotlin.cli.common.messages.AnalyzerWithCompilerReport.analyzeAndReport(AnalyzerWithCompilerReport.kt:115)
org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler.analyze(KotlinToJVMBytecodeCompiler.kt:247)
org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler.repeatAnalysisIfNeeded(KotlinToJVMBytecodeCompiler.kt:181) <2 recursive calls>
org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler.compileModules$cli(KotlinToJVMBytecodeCompiler.kt:87)
org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler.compileModules$cli$default(KotlinToJVMBytecodeCompiler.kt:43)
org.jetbrains.kotlin.cli.jvm.K2JVMCompiler.doExecute(K2JVMCompiler.kt:165)
org.jetbrains.kotlin.cli.jvm.K2JVMCompiler.doExecute(K2JVMCompiler.kt:50)
org.jetbrains.kotlin.cli.common.CLICompiler.execImpl(CLICompiler.kt:104)
org.jetbrains.kotlin.cli.common.CLICompiler.execImpl(CLICompiler.kt:48)
org.jetbrains.kotlin.cli.common.CLITool.exec(CLITool.kt:101)
org.jetbrains.kotlin.daemon.CompileServiceImpl.compile(CompileServiceImpl.kt:1523)
jdk.internal.reflect.GeneratedMethodAccessor96.invoke()
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:568)
sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:360)
sun.rmi.transport.Transport$1.run(Transport.java:200)
sun.rmi.transport.Transport$1.run(Transport.java:197)
java.security.AccessController.executePrivileged(AccessController.java:807)
java.security.AccessController.doPrivileged(AccessController.java:712)
sun.rmi.transport.Transport.serviceCall(Transport.java:196)
sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:587)
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:705)
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$$Lambda$90.0x0000000801085528.run()
java.security.AccessController.executePrivileged(AccessController.java:776)
java.security.AccessController.doPrivileged(AccessController.java:399)
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:704)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
java.lang.Thread.run(Thread.java:833)
```
```
RMI TCP Connection(565)-127.0.0.1 Parked CPU usage on sample: 0ms
jdk.internal.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:211)
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:715)
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:938)
java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:959)
org.jetbrains.kotlin.com.intellij.util.io.StorageLockContext.lockWrite(StorageLockContext.java:70)
org.jetbrains.kotlin.com.intellij.util.io.PagedFileStorage.lockWrite(PagedFileStorage.java:94)
org.jetbrains.kotlin.com.intellij.util.io.ResizeableMappedFile.lockWrite(ResizeableMappedFile.java:279)
org.jetbrains.kotlin.com.intellij.util.io.PersistentEnumeratorBase.lockStorageWrite(PersistentEnumeratorBase.java:229)
org.jetbrains.kotlin.com.intellij.util.io.PersistentEnumeratorBase.(PersistentEnumeratorBase.java:137)
org.jetbrains.kotlin.com.intellij.util.io.PersistentBTreeEnumerator.(PersistentBTreeEnumerator.java:93)
org.jetbrains.kotlin.com.intellij.util.io.PersistentEnumerator.createDefaultEnumerator(PersistentEnumerator.java:66)
org.jetbrains.kotlin.com.intellij.util.io.PersistentMapImpl.(PersistentMapImpl.java:136)
org.jetbrains.kotlin.com.intellij.util.io.PersistentMapBuilder.buildImplementation(PersistentMapBuilder.java:62)
org.jetbrains.kotlin.com.intellij.util.io.PersistentMapBuilder.build(PersistentMapBuilder.java:44)
org.jetbrains.kotlin.com.intellij.util.io.PersistentHashMap.(PersistentHashMap.java:39)
org.jetbrains.kotlin.com.intellij.util.io.PersistentHashMap.(PersistentHashMap.java:54)
org.jetbrains.kotlin.incremental.storage.CachingLazyStorage.createMap(CachingLazyStorage.kt:128)
org.jetbrains.kotlin.incremental.storage.CachingLazyStorage.getStorageIfExists(CachingLazyStorage.kt:47)
org.jetbrains.kotlin.incremental.storage.CachingLazyStorage.getKeys(CachingLazyStorage.kt:65)
org.jetbrains.kotlin.incremental.storage.LookupMap.getKeys(LookupMap.kt:56)
org.jetbrains.kotlin.incremental.TrackedLookupMap.getKeys(LookupStorage.kt:306)
org.jetbrains.kotlin.incremental.LookupStorage.getLookupSymbols(LookupStorage.kt:89)
org.jetbrains.kotlin.incremental.classpathDiff.ClasspathSnapshotShrinker$shrinkClasspath$lookupSymbols$1.invoke(ClasspathSnapshotShrinker.kt:37)
org.jetbrains.kotlin.incremental.classpathDiff.ClasspathSnapshotShrinker$shrinkClasspath$lookupSymbols$1.invoke(ClasspathSnapshotShrinker.kt:36)
org.jetbrains.kotlin.incremental.classpathDiff.ClasspathSnapshotShrinker$MetricsReporter.getLookupSymbols(ClasspathSnapshotShrinker.kt:330)
org.jetbrains.kotlin.incremental.classpathDiff.ClasspathSnapshotShrinker.shrinkClasspath(ClasspathSnapshotShrinker.kt:36)
org.jetbrains.kotlin.incremental.classpathDiff.ClasspathChangesComputer.computeClasspathChanges(ClasspathChangesComputer.kt:50)
org.jetbrains.kotlin.incremental.IncrementalJvmCompilerRunner.calculateSourcesToCompileImpl(IncrementalJvmCompilerRunner.kt:197)
org.jetbrains.kotlin.incremental.IncrementalJvmCompilerRunner.calculateSourcesToCompile(IncrementalJvmCompilerRunner.kt:135)
org.jetbrains.kotlin.incremental.IncrementalJvmCompilerRunner.calculateSourcesToCompile(IncrementalJvmCompilerRunner.kt:62)
org.jetbrains.kotlin.incremental.IncrementalCompilerRunner.tryCompileIncrementally$lambda$6$compile(IncrementalCompilerRunner.kt:207)
org.jetbrains.kotlin.incremental.IncrementalCompilerRunner.tryCompileIncrementally(IncrementalCompilerRunner.kt:247)
org.jetbrains.kotlin.incremental.IncrementalCompilerRunner.compile(IncrementalCompilerRunner.kt:116)
org.jetbrains.kotlin.daemon.CompileServiceImplBase.execIncrementalCompiler(CompileServiceImpl.kt:657)
org.jetbrains.kotlin.daemon.CompileServiceImplBase.access$execIncrementalCompiler(CompileServiceImpl.kt:105)
org.jetbrains.kotlin.daemon.CompileServiceImpl.compile(CompileServiceImpl.kt:1620)
jdk.internal.reflect.GeneratedMethodAccessor96.invoke()
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:568)
sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:360)
sun.rmi.transport.Transport$1.run(Transport.java:200)
sun.rmi.transport.Transport$1.run(Transport.java:197)
java.security.AccessController.executePrivileged(AccessController.java:807)
java.security.AccessController.doPrivileged(AccessController.java:712)
sun.rmi.transport.Transport.serviceCall(Transport.java:196)
sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:587)
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:705)
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$$Lambda$90.0x0000000801085528.run()
java.security.AccessController.executePrivileged(AccessController.java:776)
java.security.AccessController.doPrivileged(AccessController.java:399)
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:704)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
java.lang.Thread.run(Thread.java:833)
```

I can't really confirm but it seems it only happen when I have Android Studio open.

Originally reported here : https://github.com/google/dagger/issues/4233

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.