Local timestamps fields for Avro schema are not supported
- Dominant language
- Java
- Stars
- 3.1k
- Forks
- 1.6k
- Avg merge
- 3d 12h
- Merged PRs (30d)
- 33
Description
Avro schema supports the creation of local timestampe fields:
```java
LocalTimestampMillis datetimeType = LogicalTypes.localTimestampMillis();
field = new Schema.Field(attrDesc.getAttributeName(), Schema.createUnion(datetimeType.addToSchema(Schema.create(Schema.Type.LONG), Schema.create(Schema.Type.NULL)), doc, null);
```
But since there is no support in the AvroSchemaConverter.convertLogicalType method, these fields end up as BIGINT (as reported in e.g. Apache Drill).
**Reporter**: [Stefan Ziegler](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=edi.gonzales)
**Note**: *This issue was originally created as [PARQUET-2315](https://issues.apache.org/jira/browse/PARQUET-2315). Please see the [migration documentation](https://issues.apache.org/jira/browse/PARQUET-2502) for further details.*
Contributor guide
No contributing guide indexed for this repository
Research direction
Start at AvroSchemaConverter.convertLogicalType, the entry point named in the issue, and trace how localTimestampMillis logical types are currently converted. Confirm the change by checking that local timestamp fields are represented as timestamps rather than BIGINT, including the nullable union example described in the issue.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100