apache / apache/iceberg

Flink: Add Flink SQL procedures for Iceberg similarly to Apache Paimon

Open
#16,314 0 comments 0 reactions 0 assignees View on GitHub
improvement
Dominant language
Java
Stars
9.2k
Forks
3.5k
Avg merge
2d 11h
Merged PRs (30d)
132

Description

### Feature Request / Improvement

### Motivation
Apache Flink supports [CALL sql statements](https://nightlies.apache.org/flink/flink-docs-master/docs/sql/reference/utility/call/) to call a stored procedure which can perform administrative tasks. I want to implement something similar to [Paimon's procedures](https://paimon.apache.org/docs/master/flink/procedures/) in Iceberg.

The idea is to make snapshot references easier to manage directly from Flink SQL.

### Implemented procedures
- `create_tag`
- `delete_tag`
- `create_tag_from_timestamp`
- `create_branch`
- `delete_branch`
- `rollback_to`
- `rollback_to_timestamp`
- `expire_snapshots`

We can add more in the future, something like `rename_tag` or `list_snapshots` etc.

### Example usage
`CALL sys.create_tag('tableId', 'tagName', snapshotId, 'timeRetained')`

This made branch/tag workflows much more convenient in Flink SQL.

### Query engine

Flink

### Willingness to contribute

- [x] I can contribute this improvement/feature independently
- [ ] I would be willing to contribute this improvement/feature with guidance from the Iceberg community
- [ ] I cannot contribute this improvement/feature at this time

Contributor guide

Open the contributing guide

Research direction

Start with Flink's CALL statement documentation and Apache Paimon's procedure documentation, then map the listed procedures to Iceberg snapshot references. The work is done when create/delete tag and branch operations, rollback operations, and snapshot expiration are available through Flink SQL, with the example CALL form supported.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, sql
Domain
database
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.