confluentinc / confluentinc/parallel-consumer

Allow customization of the ThreadPoolExecutor

Open
#78 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
299
Forks
172
PR merge metrics
No merged PRs in 30d

Description

The code here: https://github.com/confluentinc/parallel-consumer/blob/411c925633cda63abb69e17b54ddb90a2c37afd2/parallel-consumer-core/src/main/java/io/confluent/parallelconsumer/ParallelEoSStreamProcessor.java#L206 creates a factory with some defaults, specially the default thread factory.

The problem is that on Clojure, libraries are not precompiled as `.class` files, so some Java classes and interfaces that are emitter from Clojure (for example, protocols and records) will be generated while the program is running - but, starting from Java 9, some thread factories will not share the classloader and so these classes will not be visible on the workers threads of the consumer. This makes this library hard, or even impossible, to use with Clojure.

I'll try to make an example code of this issue and link on this issue, so maybe it'll become easier to debug and even see if the problem is corrected. In the meantime, these are related problems that I found while debugging my code:

Exactly the same problem I'm having: https://stackoverflow.com/questions/55452778/parallelstream-causing-classnotfoundexception-with-jaxb-api

Work around that I'm trying to apply here: https://stackoverflow.com/questions/49113207/completablefuture-forkjoinpool-set-class-loader/57551188#57551188

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.