apache / apache/polaris

Table notification CREATE/UPDATE can fail under concurrent modification without retry

Open
#4,658 3 comments 0 reactions 0 assignees View on GitHub
stale
Dominant language
Java
Stars
2.1k
Forks
522
Avg merge
1d 17h
Merged PRs (30d)
137

Description

When Polaris receives a CREATE or UPDATE notification for an external table, the operation can fail if another process modifies the same table at the same time.

Currently:

* UPDATE fails with a concurrent modification error.
* CREATE fails if the table was created by another process before the request completes.
* In both cases, the notification fails instead of retrying.

Simply retrying the same write is not enough because the table information is read only once, making it stale after a concurrent update. A proper fix should:

1. Re-read the latest table entity on every retry.
2. Apply the new metadata location and timestamp again.
3. Retry the update with a limited number of attempts and backoff.

To keep things simple, the initial change could focus only on retrying UPDATE failures caused by concurrent modifications, since that is the most common scenario. FileIO and metadata validation would remain outside the retry loop and run only once.

Question:
Should the CREATE -> UPDATE race condition (where CREATE fails because the entity already exists) be handled in the same change, or should that be a separate follow-up? Also, is there an existing retry utility or pattern in Polaris that should be reused?

Location:
runtime/service/src/main/java/org/apache/polaris/service/catalog/iceberg/IcebergCatalog.java
(sendNotificationForTableLike)

Contributor guide

Open the contributing guide

Research direction

Start by reading runtime/service/src/main/java/org/apache/polaris/service/catalog/iceberg/IcebergCatalog.java, especially sendNotificationForTableLike. Trace how CREATE and UPDATE notifications read and write table metadata, then look for an existing retry utility or pattern in Polaris. Done means UPDATE concurrent-modification failures re-read metadata, reapply location and timestamp, and retry with bounded backoff; CREATE handling remains to be decided.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
api, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.