apache / apache/paimon

[Bug] Guard schema changes against live Global Index references

Open
#9,419 1 comment 0 reactions 0 assignees View on GitHub
bug
Dominant language
Java
Stars
3.4k
Forks
1.4k
Avg merge
1d 11h
Merged PRs (30d)
396

Description

### Search before asking

- [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.

### Paimon version

2.0

### Compute Engine

flink

### Minimal reproduce step

1. Create a table with row tracking and data evolution enabled.
2. Insert data and create a Global Index on a column.
3. Drop the indexed column.
4. Call drop_global_index for the removed column.
5. The procedure fails because the column no longer exists, while the Global Index remains referenced by the index manifest.

### What doesn't meet your expectations?

Dropping a column succeeds even when it is referenced by live Global Index files. Afterward, drop_global_index cannot resolve the removed column to its field ID, leaving the index impossible to remove through the public procedure.

Schema changes that drop, rename, or change the type of an indexed column should be rejected and instruct the user to drop the complete Global Index first. This is consistent with the existing protection in `SchemaManager#assertNotUpdatingPrimaryKeyIndexColumn` .

### Anything else?

_No response_

### Are you willing to submit a PR?

- [x] I'm willing to submit a PR!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with SchemaManager#assertNotUpdatingPrimaryKeyIndexColumn and trace the existing protection for schema changes involving index columns. Reproduce the row-tracking and data-evolution scenario with a Global Index, then inspect the schema-change and index-manifest handling. Done means dropping, renaming, or changing the type of a referenced indexed column is rejected with guidance to drop the complete Global Index first.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.