Better support for reading and writing UTC adjusted time types in parquet
- Dominant language
- Rust
- Stars
- 3.6k
- Forks
- 1.3k
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 167
Description
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
In parquet, the [TIME](https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#time) logical type has a boolean parameter for whether the values are adjusted to UTC. Arrow doesn't have any notion of time zones for time types. #6278 added a metadata key whose presence indicates that the parquet logical TIME type should be adjusted to UTC. This solution is limited in that it is not documented and the metadata key is not added on the read side unless the file was written by arrow-rs and the reader doesn't skip the arrow metadata.
**Describe the solution you'd like**
I would like this feature to be documented and for the key to be added automatically whenever the parquet TIME type is adjusted to UTC.
**Describe alternatives you've considered**
**Additional context**
Contributor guide
Research direction
Start by tracing the metadata-key implementation from #6278 through the parquet TIME read and write paths. Check how UTC adjustment is represented when Arrow metadata is skipped, then document the convention and add the key automatically for adjusted TIME values. Done means both reading and writing consistently preserve this information and coverage verifies the behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100