apache / apache/paimon

[Bug] Flink 2.0 dependency error on paimon-flink-2.0-1.2.0

Open
#6,007 2 comments 7 reactions 0 assignees View on GitHub
bug
Dominant language
Java
Stars
3.4k
Forks
1.4k
Avg merge
1d 11h
Merged PRs (30d)
396

Description

### Search before asking

- [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.

### Paimon version

paimon-flink-2.0-1.2.0

### Compute Engine

flink 2.0

### Minimal reproduce step

running flink 2.0 sinking to paimon using version 1.2.0

### What doesn't meet your expectations?

> when running on flink 2.0, paimon-flink error occurs below.

> 025-07-31 23:05:08,209 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Writer : t_cq_log (1/3) (cdbdc5e1f1366ffb024d5c8f3f73c370_306d8342cb5b2ad8b53f1be57f65bee8_0_4) switched from INITIALIZING to FAILED on 34001c4f-7bfc-4e76-858f-fd9ff90118fd @ localhost (dataPort=-1).
java.lang.NoClassDefFoundError: org/apache/flink/streaming/api/functions/sink/SinkFunction$Context
at org.apache.paimon.flink.sink.RowDataStoreWriteOperator$Factory.createStreamOperator(RowDataStoreWriteOperator.java:271) ~[paimon-flink-2.0-1.2.0.jar:1.2.0]
at org.apache.flink.streaming.api.operators.StreamOperatorFactoryUtil.createOperator(StreamOperatorFactoryUtil.java:82) ~[flink-runtime-2.0.0.jar:2.0.0]
at org.apache.flink.streaming.runtime.tasks.OperatorChain.(OperatorChain.java:214) ~[flink-runtime-2.0.0.jar:2.0.0]
at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.(RegularOperatorChain.java:60) ~[flink-runtime-2.0.0.jar:2.0.0]
at org.apache.flink.streaming.runtime.tasks.StreamTask.restoreInternal(StreamTask.java:790) ~[flink-runtime-2.0.0.jar:2.0.0]
at org.apache.flink.streaming.runtime.tasks.StreamTask.restore(StreamTask.java:770) ~[flink-runtime-2.0.0.jar:2.0.0]
at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:963) ~[flink-runtime-2.0.0.jar:2.0.0]
at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:932) ~[flink-runtime-2.0.0.jar:2.0.0]
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:756) ~[flink-runtime-2.0.0.jar:2.0.0]
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:568) ~[flink-runtime-2.0.0.jar:2.0.0]
at java.base/java.lang.Thread.run(Thread.java:829) ~[?:?]
Caused by: java.lang.ClassNotFoundException: org.apache.flink.streaming.api.functions.sink.SinkFunction$Context
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581) ~[?:?]
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) ~[?:?]
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) ~[?:?]
... 11 more

### Anything else?

I'm using gradle to build.

### Are you willing to submit a PR?

- [ ] I'm willing to submit a PR!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with org.apache.paimon.flink.sink.RowDataStoreWriteOperator.Factory.createStreamOperator at line 271 and inspect the Flink 2.0 dependency setup used by the Gradle build. Reproduce the sink failure with Flink 2.0 and Paimon 1.2.0, then verify the corrected dependency compatibility by running the same job without the NoClassDefFoundError.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering, stream-processing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.