apache / apache/iceberg

Flink: Support ALTER TABLE ... DROP PARTITION syntax

Open
#16,646 4 comments 0 reactions 0 assignees View on GitHub
improvement
Dominant language
Java
Stars
9.2k
Forks
3.5k
Avg merge
2d 11h
Merged PRs (30d)
132

Description

### Feature Request / Improvement

## Motivation
Flink already supports ```ALTER TABLE ... DROP PARTITION``` and invokes the catalog's ```dropPartition(...)``` implementation. However, the Iceberg Flink catalog does not currently implement this operation, leaving users without a SQL-level mechanism to remove all data in a partition.

### How users drop a partition today
DELETE is not supported by the Iceberg Flink catalog for batch or streaming. Users must drop into the Java/Scala Table API (table.newDelete().deleteFromRowFilter(...).commit()) or move the operation outside SQL entirely.

## Proposed feature
Add support for the ```ALTER TABLE … DROP PARTITION``` syntax, addressing the partition field name, in the same human-readable form Iceberg already writes to manifest partition paths:
```sql
ALTER TABLE events DROP PARTITION (id = 0);
ALTER TABLE events DROP PARTITION (region = 'eu', dt = '2024-01-01');
ALTER TABLE events DROP IF EXISTS PARTITION (data = 'z')
```
## Implementation sketch

- Flink: ```FlinkCatalog.dropPartition(...)``` translates the partition specification into an Iceberg row filter and invokes ```deleteFromRowFilter(...)```
- Flink's SQL parser already accepts ```ALTER TABLE … DROP [IF EXISTS] PARTITION (...)```.

## Safety
```deleteFromRowFilter``` already runs the strict-projection check at commit, so partial-file deletes are rejected by Iceberg core; the new API surface cannot silently remove rows outside the targeted partition.

### Query engine

Flink

### Willingness to contribute

- [x] I can contribute this improvement/feature independently
- [ ] I would be willing to contribute this improvement/feature with guidance from the Iceberg community
- [ ] I cannot contribute this improvement/feature at this time

Contributor guide

Open the contributing guide

Research direction

Start with FlinkCatalog.dropPartition(...) and the existing catalog partition-operation flow. Trace how the partition specification is converted into an Iceberg row filter and how deleteFromRowFilter(...) is committed; done means the listed ALTER TABLE DROP PARTITION and DROP IF EXISTS examples remove the targeted partitions safely.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.