fwcd / fwcd/kotlin-language-server

Do not iterate over excluded folders while looking for dependencies

Open
#176 1 comment 0 reactions 0 assignees View on GitHub
dependency resolution
Dominant language
Kotlin
Stars
2k
Forks
252
PR merge metrics
No merged PRs in 30d

Description

Problem:
In project I have many generated folders completely unrelated to java or kotlin.
These folders are really huge and to avoid problems with file tracking and searches I've excluded them using
```
"files.exclude": {
"build/": true,
"env/": true
}
```
With growth of this folder language server first start taking too long to start.
Unfortunately after some certain size of this folder kotlin server start crashing on startup while looking for dependencies
```
[Info - 8:01:16 PM] client Searching for dependencies in workspace root /home/xxxxxx
[Info - 8:01:19 PM] Connection to server got closed. Server will restart.
```

In certain conditions it is possible to see exception:
```
PM org.eclipse.lsp4j.jsonrpc.RemoteEndpoint fallbackResponseError
SEVERE: Internal error: java.lang.reflect.InvocationTargetException
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
at org.eclipse.lsp4j.jsonrpc.services.GenericEndpoint.lambda$null$0(GenericEndpoint.java:67)
at org.eclipse.lsp4j.jsonrpc.services.GenericEndpoint.request(GenericEndpoint.java:120)
at org.eclipse.lsp4j.jsonrpc.RemoteEndpoint.handleRequest(RemoteEndpoint.java:261)
at org.eclipse.lsp4j.jsonrpc.RemoteEndpoint.consume(RemoteEndpoint.java:190)
at org.eclipse.lsp4j.jsonrpc.json.StreamMessageProducer.handleMessage(StreamMessageProducer.java:192)
at org.eclipse.lsp4j.jsonrpc.json.StreamMessageProducer.listen(StreamMessageProducer.java:94)
at org.eclipse.lsp4j.jsonrpc.json.ConcurrentMessageProcessor.run(ConcurrentMessageProcessor.java:113)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.eclipse.lsp4j.jsonrpc.services.GenericEndpoint.lambda$null$0(GenericEndpoint.java:65)
... 11 more
Caused by: java.nio.file.FileSystemException: /home/xxx xxxxx: Too many open files
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:91)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
at sun.nio.fs.UnixFileSystemProvider.newDirectoryStream(UnixFileSystemProvider.java:427)
at java.nio.file.Files.newDirectoryStream(Files.java:457)
at java.nio.file.Files.list(Files.java:3451)
at org.javacs.kt.classpath.DefaultClassPathResolverKt.folderResolvers(DefaultClassPathResolver.kt:25)
at org.javacs.kt.classpath.DefaultClassPathResolverKt.folderResolvers(DefaultClassPathResolver.kt:33)
at org.javacs.kt.classpath.DefaultClassPathResolverKt.folderResolvers(DefaultClassPathResolver.kt:33)
at org.javacs.kt.classpath.DefaultClassPathResolverKt.folderResolvers(DefaultClassPathResolver.kt:33)
at org.javacs.kt.classpath.DefaultClassPathResolverKt.folderResolvers(DefaultClassPathResolver.kt:33)
at org.javacs.kt.classpath.DefaultClassPathResolverKt.folderResolvers(DefaultClassPathResolver.kt:33)
at org.javacs.kt.classpath.DefaultClassPathResolverKt.folderResolvers(DefaultClassPathResolver.kt:33)
at org.javacs.kt.classpath.DefaultClassPathResolverKt.folderResolvers(DefaultClassPathResolver.kt:33)
at org.javacs.kt.classpath.DefaultClassPathResolverKt.folderResolvers(DefaultClassPathResolver.kt:33)
at org.javacs.kt.classpath.DefaultClassPathResolverKt.folderResolvers(DefaultClassPathResolver.kt:33)
at org.javacs.kt.classpath.DefaultClassPathResolverKt.folderResolvers(DefaultClassPathResolver.kt:33)
at org.javacs.kt.classpath.DefaultClassPathResolverKt.folderResolvers(DefaultClassPathResolver.kt:33)
at org.javacs.kt.classpath.DefaultClassPathResolverKt.folderResolvers(DefaultClassPathResolver.kt:33)
at org.javacs.kt.classpath.DefaultClassPathResolverKt.folderResolvers(DefaultClassPathResolver.kt:33)
at org.javacs.kt.classpath.DefaultClassPathResolverKt.folderResolvers(DefaultClassPathResolver.kt:33)
at org.javacs.kt.classpath.DefaultClassPathResolverKt.folderResolvers(DefaultClassPathResolver.kt:33)
at org.javacs.kt.classpath.DefaultClassPathResolverKt.folderResolvers(DefaultClassPathResolver.kt:33)
at org.javacs.kt.classpath.DefaultClassPathResolverKt.folderResolvers(DefaultClassPathResolver.kt:33)
at org.javacs.kt.classpath.DefaultClassPathResolverKt.workspaceResolvers(DefaultClassPathResolver.kt:18)
at org.javacs.kt.classpath.DefaultClassPathResolverKt.access$workspaceResolvers(DefaultClassPathResolver.kt:1)
at org.javacs.kt.classpath.DefaultClassPathResolverKt$defaultClassPathResolver$1.invoke(DefaultClassPathResolver.kt:12)
at org.javacs.kt.classpath.DefaultClassPathResolverKt$defaultClassPathResolver$1.invoke(DefaultClassPathResolver.kt)
at kotlin.sequences.FlatteningSequence$iterator$1.ensureItemIterator(Sequences.kt:277)
at kotlin.sequences.FlatteningSequence$iterator$1.hasNext(Sequences.kt:265)
at org.javacs.kt.classpath.ClassPathResolverKt.getJoined(ClassPathResolver.kt:66)
at org.javacs.kt.classpath.DefaultClassPathResolverKt.defaultClassPathResolver(DefaultClassPathResolver.kt:12)
at org.javacs.kt.CompilerClassPath.refresh(CompilerClassPath.kt:20)
at org.javacs.kt.CompilerClassPath.refresh$default(CompilerClassPath.kt:18)
at org.javacs.kt.CompilerClassPath.addWorkspaceRoot(CompilerClassPath.kt:64)
at org.javacs.kt.KotlinLanguageServer.initialize(KotlinLanguageServer.kt:76)
... 16 more
```

I think it simplest way is not trying to iterate over excluded folders.

sys limit for this could not solve the problem, as it is already large enough (in my case 1.5M)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start in DefaultClassPathResolver.kt, especially the folderResolvers and workspaceResolvers paths shown in the stack trace, and trace how the workspace root is traversed during dependency discovery. Reproduce with large build/ or env/ directories excluded through files.exclude; done means those directories are skipped and startup no longer exhausts file descriptors or crashes.

Written by the indexing model from the issue text.

Assessment

Tech stack
kotlin
Domain
devtools
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.