ktorio / ktorio/ktor

Have Ktor automatically load platform-specific native transport dependencies for Netty, if available

Open
#1,672 1 comment 0 reactions 0 assignees View on GitHub
feature Server
Dominant language
Kotlin
Stars
14.5k
Forks
1.3k
Avg merge
2d 13h
Merged PRs (30d)
49

Description

**NOTE**: this issue at first glance may seem a duplicate of #1122, but that's not the case. In the other issue, Ktor is enhanced to automatically load the native transports, if found in the classpath. This new issue however, is about automatically including the platform-specific transitive dependencies (the ones with platform-specific classifiers) when including the Ktor dependencies in a project.

**Subsystem**
N/A

**Is your feature request related to a problem? Please describe.**
It has been reported multiple times that Netty logs a number of exceptions to the console at the DEBUG level while trying to load a native transport library on application startup. It then fails to do so and falls back to the standard multiplatform NIO transport, which results in somewhat reduced performance.

Currently, I'm using the following workaround in the `dependencies` block of the `build.gradle.kts` file (assuming Gradle with Kotlin DSL), to smartly load the platform-specific native transport dependencies, if available for the underlying platform:

```kotlin
dependencies {
// Your project-specific dependencies here
// implementation("...")
// implementation("...")
// ...etc.

fun determineTransitiveNettyVersion(): String? {
var determinedNettyVersion: String? = null
components.all {
allVariants {
withDependencies {
forEach { dep ->
if (dep.group == "io.netty" && dep.name == "netty-common") {
dep.version {
determinedNettyVersion = requiredVersion
}
}
}
}
}
}
return determinedNettyVersion
}

// Try to use a native transport for Netty, for increased performance, if available for the current OS type.
// See https://netty.io/wiki/native-transports.html
val currentOS = org.gradle.nativeplatform.platform.internal.DefaultNativePlatform.getCurrentOperatingSystem()
val currentArch = org.gradle.nativeplatform.platform.internal.DefaultNativePlatform.getCurrentArchitecture()
if (currentArch.isAmd64) {
val transitiveNettyVersion = determineTransitiveNettyVersion()
if (currentOS.isLinux) {
implementation("io.netty:netty-transport-native-epoll:$transitiveNettyVersion:linux-x86_64")
} else if (currentOS.isMacOsX || currentOS.isFreeBSD) {
implementation("io.netty:netty-transport-native-kqueue:$transitiveNettyVersion:osx-x86_64")
} else {
println(
"No Netty native transport available for OS \"${currentOS.name}\". Netty will fall back to using the " +
"NIO transport."
)
}
} else {
println(
"No Netty native transport available for architecture \"${currentArch.name}\". Netty will fall back " +
"to using the NIO transport."
)
}
}
```

**Describe the solution you'd like**
It would be convenient if Ktor could somehow handle this automatically, if possible. To be honest however, I'm not sure if there is a solution that would automagically work for transitive dependencies with classifiers, let alone one that would work in both Gradle and Maven projects.

Bonus points if there is a way to suppress the remaining debug-level exceptions that Netty will still always throw, since it always first tries multiple unsuccessful alternative ways of loading the library, even if it ultimately succeeds. But this might be better dealt with upstream in the Netty project.

And finally, log only a single message about native transport not being available when the loading of such a library was ultimately unsuccessful (for instance, when the application is deployed on a platform for which no native transport is available in Maven Central, JCenter, etc). This instead of the "successfully loaded..." message that Netty normally logs on success.

**Motivation to include to ktor**
It saves the hassle of configuring something that basically everybody wants: more optimal performance and resource usage efficiency for free and without any effort, if available.

**Alternative proposal**

If this proves infeasible, then please extend the Ktor documentation with instructions on how to add the necessary platform-specific native transport dependencies to Gradle or Maven projects. Feel free to use the workaround code I shared above.

Contributor guide

Open the contributing guide

Research direction

Start with the Gradle Kotlin DSL dependencies block and the Netty native transport setup described in the issue. Investigate whether platform-specific transitive dependencies can be handled for both Gradle and Maven, and review the proposed documentation fallback. Done would require a decided approach for automatic loading or documented configuration instructions.

Written by the indexing model from the issue text.

Assessment

Tech stack
kotlin
Domain
build-system, networking
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.