spring-projects / spring-projects/spring-boot

Automatically configure the common ForkJoinPool to use Boot's LaunchedClassLoader

Open
#39,843 9 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

type: enhancement
Dominant language
Java
Stars
81.5k
Forks
42.7k
Avg merge
2d 4h
Merged PRs (30d)
65

Description

Context

I created this issue as a bug report or enhancement proposal - depending on how would you classify current behaviour.

I have a spring application that I am building using "org.springframework.boot gradle" plugin. This plugin builds Executable Jar and War as described in documentation:
https://docs.spring.io/spring-boot/docs/current/reference/html/executable-jar.html

Problem

Executable Jar uses custom class loader: org.springframework.boot.loader.launch.LaunchedClassLoader when running the application.
This class loader is not propagated to the common ForkJoinPool, which uses system class loader by default.

Take a code like that:

IntStream.rangeClosed(0, 4)
    .parallel()
    .forEach(i -> System.out.println(Thread.currentThread().getName() + " " + Thread.currentThread().getContextClassLoader()));

It will produce following output:

ForkJoinPool.commonPool-worker-1 jdk.internal.loader.ClassLoaders$AppClassLoader@33909752
ForkJoinPool.commonPool-worker-2 jdk.internal.loader.ClassLoaders$AppClassLoader@33909752
ForkJoinPool.commonPool-worker-1 jdk.internal.loader.ClassLoaders$AppClassLoader@33909752
http-nio-8080-exec-1 TomcatEmbeddedWebappClassLoader
  context: ROOT
  delegate: true
----------> Parent Classloader:
org.springframework.boot.loader.launch.LaunchedClassLoader@1a6c5a9e

We have 4 tasks to execute in parallel. For such execution, java uses commom ForkJoinPool.

One of the tasks executed on current thread (http-nio-8080-exec-1) and it sees "correct" class loader: LaunchedClassLoader.
Other three tasks executed on separate threads, that see "incorrect", system class loader: AppClassLoader

This causes issues if we try to execute in parallel piece of code that needs to have access to the "proper" class loader.

This behaviour is even described in the documentation: https://docs.spring.io/spring-boot/docs/current/reference/html/executable-jar.html#appendix.executable-jar.restrictions

System classLoader: Launched applications should use Thread.getContextClassLoader() when loading classes (most libraries and frameworks do so by default). Trying to load nested jar classes with ClassLoader.getSystemClassLoader() fails.

The problem is that the the class that we are considering here - common ForkJoinPool - is a big part of JDK itself

Possible Fix / Enhancement

Common ForkJoinPool can be configured to use different ThreadFactory (by setting java.util.concurrent.ForkJoinPool.common.threadFactory system property) - for example, custom ThreadFactory that returns threads with LaunchedClassLoader

Workarounds

Configure custom thread factory

You can create your custom thread factory like so:

public class MyForkJoinWorkerThreadFactory implements ForkJoinWorkerThreadFactory {
    @Override
    public final ForkJoinWorkerThread newThread(ForkJoinPool pool) {
        return new MyForkJoinWorkerThread(pool);
    }

    private static class MyForkJoinWorkerThread extends ForkJoinWorkerThread {
        private MyForkJoinWorkerThread(final ForkJoinPool pool) {
            super(pool);
            setContextClassLoader(Thread.currentThread().getContextClassLoader());
        }
    }
} 

You can set up system property java.util.concurrent.ForkJoinPool.common.threadFactory=foo.bar.MyForkJoinWorkerThreadFactory to make common ForkJoinPool use this thread factory.

Problem: ForkJoinPool will use system class loader to find foo.bar.MyForkJoinWorkerThreadFactory, so it must be part of spring boot launcher class path

Don't use common ForkJoinPool

We could use custom ForkJoinPool with custom ForkJoinWorkerThreadFactory like

try(ForkJoinPool pool = new ForkJoinPool(4, new MyForkJoinWorkerThreadFactory(), null, false)) {
    pool.submit(() -> IntStream.rangeClosed(0, 4).parallel()
        .forEach(i -> System.out.println(Thread.currentThread().getName() + " " + Thread.currentThread().getContextClassLoader())););
}

Problems:

  • It's very verbose, you need to wrap all you application entry points in custom ForkJoinPool
  • Common ForkJoinPool implementation is a bit different then ForkJoinPool. Namely, it always uses current thread as one of the worker threads. This functionality is very useful in some contexts, and you can't achieve it using custom ForkJoinPool

Don't use Executable Jar format

You can try building jar for you spring application withou using Executable Jar (without the launcher). Documentation even lists some alternative methods in Alternative Single Jar Solutions

Problem: Building fat jar for spring application is quite hard. I tried using Gradle Shadow Plugin, but it is hard to correctly merge every necessery file. I didn't found any (up to date) solution that would worker

Unpack fat jar

You can also unpack fat jar created by spring boot plugin and run your application manually, without the launcher (as described in https://stackoverflow.com/questions/58746223/are-there-caveats-to-not-using-the-spring-boot-classloader-in-production)

$ jar -xf myapp.jar
$ java -cp "BOOT-INF/classes:BOOT-INF/lib/*" com.example.MyApplication

Problem: Won't work in environments where you have to provide executable jar file

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with Spring Boot's LaunchedClassLoader and the common ForkJoinPool thread-factory configuration described in the issue. Determine how executable-jar startup could apply the correct context class loader to common-pool workers without application-specific setup. Done means parallel tasks launched in an executable jar observe the LaunchedClassLoader, with regression coverage for this behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, spring-boot
Domain
backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.