[Improvement]: When manually setting table.event-time-field to a field, it should widen its data file mertic length
- Dominant language
- Java
- Stars
- 1.2k
- Forks
- 395
- Avg merge
- 4d 10h
- Merged PRs (30d)
- 33
Description
### Search before asking
- [X] I have searched in the [issues](https://github.com/NetEase/arctic/issues?q=is%3Aissue) and found no similar issues.
### What would you like to be improved?
The current default length of the iceberg table for metadata metric is 'write.metadata.metrics.default' = 'truncate(16)'. meaning that data larger than 16 will have a loss of precision.
When we set table.event-time-field we often choose a time field in the format yyyy-MM-dd HH:mm:ss, obviously this string is longer than 16, then there will be a loss of precision when parsing the upperbounds and lowerbounds of the datafile.
From the user's point of view, most of the cases will not be able to change 'write.metadata.metrics.default' to a larger value or to 'full', so it is necessary to help the user to set this configuration.
### How should we improve?
AMS should set global default configuration such as 'write.metadata.metrics.default' to 'truncate(32)' when creating a table, or just for the fields set in the table.event-time-field configuration mertric changes
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Subtasks
_No response_
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
Contributor guide
Research direction
Start by tracing AMS table creation and handling of the table.event-time-field configuration, then inspect how the Iceberg metadata metric default is applied. Compare the global and field-specific approaches described in the issue and add focused coverage for event-time fields longer than 16 characters. Done means upper and lower bounds retain the required precision without users changing the default configuration.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering, databases
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 50/100