apache / apache/iceberg

Core: Expose commit retry exhaustion reason in failure messages

Open
#16,744 0 comments 0 reactions 0 assignees View on GitHub
improvement
Dominant language
Java
Stars
9.2k
Forks
3.5k
Avg merge
2d 11h
Merged PRs (30d)
132

Description

### Feature Request / Improvement

Commit failures caused by exhausted `commit.retry.*` backoff currently rethrow the underlying `CommitFailedException` without indicating why the retry loop stopped.

This makes it hard for clients/operators to know which table property to tune:
- `commit.retry.num-retries`
- `commit.retry.total-timeout-ms`
- potentially `commit.retry.min-wait-ms` / `commit.retry.max-wait-ms`

## Proposal

Expose whether commit retry stopped because the retry attempt budget was exhausted, the total retry timeout was exceeded, or both.

The implementation should avoid making generic retry utilities aware of Iceberg commit property names. A good direction would be:
- classify retry exhaustion in `Tasks`
- preserve the original exception as the cause
- translate the retry exhaustion reason into commit-specific guidance at commit call sites

## Expected behavior

When a commit fails after exhausting retries, the final exception message should include actionable guidance such as:
- increase `commit.retry.num-retries` when the attempt limit is reached
- increase `commit.retry.total-timeout-ms` when the elapsed retry timeout is reached
- consider adjusting wait properties if retry pacing is relevant

## Motivation

This gives clients better visibility into whether failed commits need more retry attempts or a longer retry window, instead of requiring log inspection or guesswork.

### Query engine

None

### Willingness to contribute

- [x] I can contribute this improvement/feature independently
- [ ] I would be willing to contribute this improvement/feature with guidance from the Iceberg community
- [ ] I cannot contribute this improvement/feature at this time

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.