microsoft / microsoft/multilspy
Kotlin hover tests flaky: KLS NullPointerException when Gradle classpath resolution fails in CI
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 610
- Forks
- 111
- PR merge metrics
- No merged PRs in 30d
Description
Problem
The Kotlin Language Server (KLS v1.3.13) intermittently throws a NullPointerException on textDocument/hover requests when it cannot resolve the project classpath via Gradle or Maven. This causes test_multilspy_kotlin_hover and test_multilspy_kotlin_sync_hover to fail non-deterministically in CI.
Failing tests:
tests/multilspy/test_multilspy_kotlin.py::test_multilspy_kotlin_hovertests/multilspy/test_sync_multilspy_kotlin.py::test_multilspy_kotlin_sync_hover
Both tests are currently marked xfail(strict=False) in #133.
Root cause
On GitHub Actions runners, /usr/bin/gradle and /usr/bin/mvn exist but cannot be executed by the KLS Java process (error=13, Permission denied), even after sudo chmod a+x. The chmod fixes shell access but not JVM ProcessBuilder access — likely a runner security policy issue.
When classpath resolution fails, the KLS falls back to kotlinc stdlib only. With incomplete type information, hover requests on certain symbols trigger a NullPointerException inside the KLS:
Internal error: java.lang.NullPointerException
java.util.concurrent.CompletionException: java.lang.NullPointerException
at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(Unknown Source)
at java.base/java.util.concurrent.CompletableFuture.completeThrowable(Unknown Source)
at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(Unknown Source)
This surfaces as multilspy.lsp_protocol_handler.server.Error: Internal error. (-32603).
Evidence that it's flaky (not consistent)
From CI run #24214120325:
- Python 3.10: all Kotlin tests passed (both async and sync hover)
- Python 3.14: all Kotlin tests passed
- Python 3.12: async hover passed, sync hover failed
Same Gradle permission errors on all three, but the NullPointerException only triggers non-deterministically.
Possible fixes
- Upstream KLS fix: The KLS should handle missing classpath gracefully on hover instead of throwing NPE. This is a bug in fwcd/kotlin-language-server.
- Use a test repo that doesn't need Gradle: The current test repo is the KLS source itself (a Gradle project). A simpler Kotlin project with no build system dependency would avoid the classpath issue entirely.
- Runner-level fix: Figure out why JVM
ProcessBuildercan't exec/usr/bin/gradleafterchmod a+xon GitHub runners.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with tests/multilspy/test_multilspy_kotlin.py::test_multilspy_kotlin_hover and tests/multilspy/test_sync_multilspy_kotlin.py::test_multilspy_kotlin_sync_hover, then review the xfail added in #133 and the linked CI run. Determine whether the fix belongs in the test project, runner setup, or upstream KLS; done means the Kotlin hover tests run deterministically without the KLS NullPointerException.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions, kotlin, python
- Domain
- ci-cd, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100