apache / apache/iceberg

Make Metadata Timestamp Clock Skew Configurable

Open
#17,285 0 comments 0 reactions 0 assignees View on GitHub
improvement
Dominant language
Java
Stars
9.2k
Forks
3.5k
Avg merge
2d 11h
Merged PRs (30d)
132

Description

### Feature Request / Improvement

Background

Iceberg currently validates metadata timestamps using a hardcoded one-minute tolerance (TableMetadata.ONE_MINUTE) when constructing new table metadata during a commit.

This validation helps detect situations where metadata timestamps are significantly ahead of the local system clock, which could indicate incorrect system time or metadata corruption.

However, in deployments where multiple compute engines (for example, Spark, Flink, Trino, or custom applications) operate on the same Iceberg table from different clusters, temporary clock skew greater than one minute can occur despite otherwise healthy deployments. In such cases, valid commits may fail because the metadata timestamp appears to be too far in the future relative to the committing process.

Although proper clock synchronization (for example, via NTP) is recommended, operators may still encounter larger clock skew in distributed or multi-region environments.

Problem

The allowable clock skew is currently hardcoded as one minute and cannot be configured.

As a result:

Valid commits may fail when clocks differ by more than one minute.
Operators have no mechanism to tune this behavior for their deployment.
The only current workaround is improving clock synchronization across all participating systems.
Proposed Solution

Introduce a JVM system property to configure the maximum allowable clock skew used during metadata timestamp validation.

System Property
iceberg.commit.allowed-clock-skew-ms
Default Value
60000

(Existing behavior remains unchanged.)

Example
-Diceberg.commit.allowed-clock-skew-ms=300000

This would allow metadata timestamps to be up to five minutes ahead of the local system clock before the commit is rejected.

Behavior

The validation logic would determine the allowed clock skew as follows:

Read the JVM system property iceberg.commit.allowed-clock-skew-ms.
If the property is present and valid, use its value.
Otherwise, fall back to the current default of 60,000 milliseconds.

This proposal does not change the default behavior for existing deployments.

Compatibility

This proposal is fully backward compatible.

Existing applications continue to use the current one-minute tolerance.
No table metadata changes are required.
No catalog changes are required.
No API changes are required.
Future Work

A future enhancement could introduce catalog-level configuration, allowing operators to configure the allowable clock skew once per catalog instead of per JVM.

One possible configuration hierarchy would be:

Catalog property

System property

Default (60000 ms)

The current proposal intentionally limits the scope to a JVM system property to minimize implementation complexity while addressing the immediate operational need.

### Query engine

None

### Willingness to contribute

- [x] I can contribute this improvement/feature independently
- [ ] I would be willing to contribute this improvement/feature with guidance from the Iceberg community
- [ ] I cannot contribute this improvement/feature at this time

Contributor guide

Open the contributing guide

Research direction

Start at TableMetadata.ONE_MINUTE and trace the metadata timestamp validation used when constructing new table metadata during a commit. Check nearby tests for this validation, then verify that iceberg.commit.allowed-clock-skew-ms accepts valid values while invalid or absent values preserve the 60,000 ms default.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases, distributed-systems
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.