apache / apache/pulsar

[Bug] ClassNotFoundException error occurred when using Function development in the kubernetes cluster

Open
#22,513 1 comment 0 reactions 0 assignees View on GitHub
type/bug
Dominant language
Java
Stars
15.3k
Forks
3.8k
Avg merge
1d 14h
Merged PRs (30d)
160

Description

### Search before asking

- [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.

### Read release policy

- [X] I understand that unsupported versions don't get bug fixes. I will attempt to reproduce the issue on a supported version of Pulsar client and Pulsar broker.

### Version

- kubernetes: 1.28
- jdk: 17
- pulsar: 3.2.1
- pulsar-functions-pulsarctl-java-runner: 3.2.1.2

### Minimal reproduce step

1. Develop using the Extended Pulsar Functions SDK for Java and connect to the Redis cluster
2. Edit yaml file
3. kubectl apply -f function.yaml

### What did you expect to see?

The function can connect to the Redis cluster and work properly.

### What did you see instead?

```java
2024-04-15T06:18:32,716+0000 [public/default/demo-function-las-auth-0] ERROR org.apache.pulsar.functions.instance.JavaInstanceRunnable - Sink open produced uncaught exception:
java.lang.ClassNotFoundException: com.alibaba.fastjson2.JSONObject
at java.net.URLClassLoader.findClass(URLClassLoader.java:445) ~[?:?]
at java.lang.ClassLoader.loadClass(ClassLoader.java:592) ~[?:?]
at java.lang.ClassLoader.loadClass(ClassLoader.java:525) ~[?:?]
at org.apache.pulsar.common.util.Reflections.loadClass(Reflections.java:310) ~[io.streamnative-pulsar-common-3.2.1.2.jar:3.2.1.2]
at org.apache.pulsar.functions.sink.PulsarSink.initializeSchema(PulsarSink.java:437) ~[io.streamnative-pulsar-functions-instance-3.2.1.2.jar:3.2.1.2]
at org.apache.pulsar.functions.sink.PulsarSink.open(PulsarSink.java:362) ~[io.streamnative-pulsar-functions-instance-3.2.1.2.jar:3.2.1.2]
at org.apache.pulsar.functions.instance.JavaInstanceRunnable.setupOutput(JavaInstanceRunnable.java:1070) ~[io.streamnative-pulsar-functions-instance-3.2.1.2.jar:?]
at org.apache.pulsar.functions.instance.JavaInstanceRunnable.setup(JavaInstanceRunnable.java:261) ~[io.streamnative-pulsar-functions-instance-3.2.1.2.jar:?]
at org.apache.pulsar.functions.instance.JavaInstanceRunnable.run(JavaInstanceRunnable.java:305) ~[io.streamnative-pulsar-functions-instance-3.2.1.2.jar:?]
at java.lang.Thread.run(Thread.java:840) ~[?:?]
2024-04-15T06:18:32,725+0000 [public/default/demo-function-las-auth-0] ERROR org.apache.pulsar.functions.instance.JavaInstanceRunnable - [public/default/demo-function-las-auth:0] Uncaught exception in Java Instance
java.lang.ClassNotFoundException: com.alibaba.fastjson2.JSONObject
at java.net.URLClassLoader.findClass(URLClassLoader.java:445) ~[?:?]
at java.lang.ClassLoader.loadClass(ClassLoader.java:592) ~[?:?]
at java.lang.ClassLoader.loadClass(ClassLoader.java:525) ~[?:?]
at org.apache.pulsar.common.util.Reflections.loadClass(Reflections.java:310) ~[io.streamnative-pulsar-common-3.2.1.2.jar:3.2.1.2]
at org.apache.pulsar.functions.sink.PulsarSink.initializeSchema(PulsarSink.java:437) ~[io.streamnative-pulsar-functions-instance-3.2.1.2.jar:3.2.1.2]
at org.apache.pulsar.functions.sink.PulsarSink.open(PulsarSink.java:362) ~[io.streamnative-pulsar-functions-instance-3.2.1.2.jar:3.2.1.2]
at org.apache.pulsar.functions.instance.JavaInstanceRunnable.setupOutput(JavaInstanceRunnable.java:1070) ~[io.streamnative-pulsar-functions-instance-3.2.1.2.jar:?]
at org.apache.pulsar.functions.instance.JavaInstanceRunnable.setup(JavaInstanceRunnable.java:261) ~[io.streamnative-pulsar-functions-instance-3.2.1.2.jar:?]
at org.apache.pulsar.functions.instance.JavaInstanceRunnable.run(JavaInstanceRunnable.java:305) ~[io.streamnative-pulsar-functions-instance-3.2.1.2.jar:?]
at java.lang.Thread.run(Thread.java:840) ~[?:?]
```

### Anything else?

```yaml
apiVersion: compute.functionmesh.io/v1alpha1
kind: Function
metadata:
name: demo-function-las-auth
namespace: pulsar
spec:
image: streamnative/pulsar-functions-pulsarctl-java-runner:3.2.1.2 # using java function runner
className: com.audit.function.handler.LasAuthFunction
forwardSourceMessageProperty: true
maxPendingAsyncRequests: 1000
replicas: 1
maxReplicas: 5
logTopic: persistent://demo-tenants/log-namespace/function-las-auth-logs
input:
topics:
- persistent://demo-tenants/log-namespace/las-auth
typeClassName: "[Ljava.lang.String;"
output:
topic: persistent://demo-tenants/log-namespace/las-auth-done
typeClassName: "com.alibaba.fastjson2.JSONObject"
funcConfig:
redis.url: redis-cluster.redis-cluster.svc.cluster.local
redis.password: ******
secretsMap:
"redis-ca-certs":
path: "redis-ca-certs"
key: "ca.crt"
"redis-tls-cert":
path: "redis-tls-cert"
key: "tls.crt"
"redis-tls-key":
path: "redis-tls-key"
key: "tls.key"
pulsar:
pulsarConfig: "demo-pulsar"
authSecret: "pulsar-auth"
tlsConfig:
enabled: true
allowInsecure: true
hostnameVerification: false
certSecretName: pulsar-tls-proxy
certSecretKey: ""
resources:
requests:
cpu: "2"
memory: 4G
limits:
cpu: "2"
memory: 4G
java:
extraDependenciesDir: random-dir/
jar: demo-function-1.1-SNAPSHOT.jar # the package will download as this filename.
log:
level: "debug"
jarLocation: function://demo-tenants/log-namespace/demo-las-auth-function@1.1 # function package URL
javaOpts:
- "--add-opens java.base/sun.net=ALL-UNNAMED"
- "--add-opens java.management/sun.management=ALL-UNNAMED"
autoAck: true
---
apiVersion: v1
kind: ConfigMap
metadata:
name: demo-pulsar
namespace: pulsar
data:
webServiceURL: https://pulsar-proxy.pulsar.svc.cluster.local:443
brokerServiceURL: pulsar+ssl://pulsar-proxy.pulsar.svc.cluster.local:6651
---
apiVersion: v1
metadata:
name: pulsar-auth
namespace: pulsar
data:
clientAuthenticationPlugin: ******
clientAuthenticationParameters: ******
kind: Secret
type: Opaque
```

### Are you willing to submit a PR?

- [ ] I'm willing to submit a PR!

Contributor guide

Open the contributing guide

Research direction

Start with the supplied Function YAML and trace the reported failure through PulsarSink.java, Reflections.java, and JavaInstanceRunnable.java. Reproduce the Java function deployment with the stated runner and output type, then determine what prevents com.alibaba.fastjson2.JSONObject from loading. Done means the function starts and connects to Redis without the ClassNotFoundException.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, kubernetes, redis
Domain
backend, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.