AbsaOSS / AbsaOSS/enceladus

Support enhanced timestamp logical types in standardization schema

Open
#174 0 comments 0 reactions 1 assignee Claimed by @benedeki View on GitHub
feature Menas priority: low
Dominant language
Scala
Stars
33
Forks
16
PR merge metrics
No merged PRs in 30d

Description

## Background
Currently, the way we infer the timezone is if the schema's metadata provides a timezone in the timestamp pattern or an additional timezone field. If no such information is provided we assume it is already UTC.

## Feature
It would be helpful to users to make the distinction between `timezoned` and `presumed-UTC` timestamps in the UI.
However, it is important to note that we shouldn't enforce this in the actual schema file as that would make the schemas we use and the actual `SparkStruct` schemas would no longer be compatible.

## Proposed Solution
Once a schema editor is implemented (#31) we can incorporate these two logical types to make it easier for users to provide the proper timezone information.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.