apache / apache/polaris

Azure cleanup can drop tasks when ADLSFileIO suppresses deletion failures

Open
#5,482 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
2.1k
Forks
522
Avg merge
1d 22h
Merged PRs (30d)
137

Description

### Describe the bug

Azure batch file cleanup can still report success and remove its persisted task when an ADLS deletion fails.

PR #5421 restores Polaris retries for failures reported by `FileIO`, but Iceberg 1.11.0's `ADLSFileIO.deleteFile` catches `RuntimeException` and only logs it. Polaris therefore cannot observe the failure, retry it, or preserve the task after retry exhaustion.

### To Reproduce

1. Configure Polaris cleanup with Iceberg's `ADLSFileIO`.
2. Trigger batch metadata cleanup, such as `DROP TABLE PURGE`.
3. Make the Azure delete operation fail with a runtime storage or network error.
4. Observe that `ADLSFileIO.deleteFile` returns normally after logging the error.

### Actual Behavior

Polaris treats the deletion as successful and can remove the persisted cleanup task while the file remains in ADLS.

### Expected Behavior

The Azure deletion failure should reach Polaris so its bounded retry loop runs. If retries are exhausted, the cleanup task should remain persisted for later recovery.

### Additional context

- Follow-up to #5420 and #5421, identified in [PR review discussion](https://github.com/apache/polaris/pull/5421#discussion_r3961531860).
- Iceberg 1.11.0 catches the error in [`ADLSFileIO.deleteFile`](https://github.com/apache/iceberg/blob/apache-iceberg-1.11.0/azure/src/main/java/org/apache/iceberg/azure/adlsv2/ADLSFileIO.java#L109-L119).
- Related Iceberg logging issue: apache/iceberg#13518. That issue did not change failure propagation.
- The likely fix belongs in Iceberg's Azure `FileIO`; Polaris should then consume a release containing it.

### System information

- Polaris: current `main` plus #5421
- Iceberg: 1.11.0
- Object storage: Azure Data Lake Storage through `ADLSFileIO`

Contributor guide

Open the contributing guide

Research direction

Start with Iceberg 1.11.0's ADLSFileIO.deleteFile at the linked lines, then read Polaris's cleanup retry behavior from PR #5421. Reproduce a runtime Azure deletion failure and verify it reaches Polaris, retries within the bounded loop, and leaves the persisted task after retry exhaustion.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, java
Domain
backend, cloud
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.