Writing UTC timestamp to an iceberg table fails
- Dominant language
- Rust
- Stars
- 1.4k
- Forks
- 567
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 93
Description
### Apache Iceberg Rust version
None
### Describe the bug
Writing UTC timestamp to an iceberg table fails in the arrow writer. [A discussion was had ](https://github.com/apache/arrow-rs/pull/9994) in arrow-rs if the logic should live there to slacken this and the suggestion was to handle this in application code similar to what datafusion does for this exact problem.
```
ArrowError(
"Incompatible type. Field 'timestamp' has type Timestamp(µs, \"+00:00\"), array has type Timestamp(µs, \"UTC\")",
),
```
### To Reproduce
_No response_
### Expected behavior
Don't fail, these two types are identical semantically
### Willingness to contribute
I can contribute a fix for this bug independently
Contributor guide
Research direction
Start in the Iceberg arrow writer where the incompatible timestamp types are rejected, then review the linked arrow-rs discussion and DataFusion's handling of the same problem. Confirm the expected behavior by reproducing a write with UTC timestamps and verify that Timestamp(µs, "UTC") and Timestamp(µs, "+00:00") are accepted as semantically equivalent.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100