apache / apache/paimon

[Feature] Avoid deprecated APIs removed in Flink 2.0 Preview

Open
#4,442 3 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Java
Stars
3.4k
Forks
1.4k
Avg merge
1d 11h
Merged PRs (30d)
396

Description

### Search before asking

- [X] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.

### Motivation

Flink 2.0 Preview has been released, and Paimon needs to make itself compatible with 2.0 Preview now, to be better prepared for the Flink 2.0 in future.

### Solution

A bunch of public APIs that had been marked as `@Deprecated` are now removed in Flink 2.0 Preview. Paimon needs to replace usages of these deprecated APIs to the latest alternatives.

Subtasks | Related PR | Status
-- | -- | --
[hotfix] Wait for consumer reset before job close | #4578 | Merged
[cdc] Update flink dependency to 1.20 | #4580 | Merged
Adopt open(OpenContext) in RichFunction | #4581 | Merged
Adopt getTaskInfo() when acquiring parallelism info | #4583 | Merged
Avoid deprecated usages about Configuration | #4584 | Merged
Avoid deprecated DataStreamUtils | #4590 | Merged
Remove deprecated TestEnvironment | #4590 | Merged
Replace deprecated flink Time with java Duration | #4590 | Merged
Avoid deprecated SingleThreadMultiplexSourceReaderBase constructor | #4590 | Merged
Avoid deprecated FileSystem#getKind | #4590 | Merged
Avoid deprecated SetupableStreamOperator | #4591 | Merged
Avoid deprecated usage on Table API, including TableSchema, DataType and DescriptorProperties | #4611 | Merged
Replace legacy SinkFunction with v2 Sink | #4612 | Merged
Replace legacy SourceFunction with v2 Source | #4614 | Under Review
Avoid relying on format of table description | #4590 | Merged
Avoid deprecated sql syntax |   | Waiting for external bugfix: [FLINK-36783](https://issues.apache.org/jira/browse/FLINK-36783)
Change MultipleParameterTool package name |   | Waiting for Flink 2.0 formal release
Create profile for Flink 2.0 using Java 11 |   | Waiting for Flink 2.0 formal release
Avoid external legacy SourceFunction/SinkFunction in Flink Kafka Connector |   | Waiting for external work: Flink Kafka Connector avoids legacy sink function

### Anything else?

Flink 2.0 still have more work to do with its internal implementation, and only dealt with its public APIs in 2.0 Preview. So if Paimon has been using Flink's public methods that are not marks as `@Public` or `@PublicEvolving`, the usages might still be compatible with 2.0 Preview for now, but should also be updated to alternatives as early as possible.

### Are you willing to submit a PR?

- [X] I'm willing to submit a PR!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reviewing the remaining subtasks and the linked related PRs to identify which Flink 2.0 compatibility work is still open. Done means the outstanding deprecated or removed Flink API usages are replaced, while items waiting on external Flink work remain tracked.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.