apache / apache/iceberg

[Connect] Currently fields of type INT64 with logical name Timestamp are created as iceberg type TimestampType.withZone(), which is not fully supported by Athena

Open
#15,761 0 comments 0 reactions 0 assignees View on GitHub
improvement
Dominant language
Java
Stars
9.2k
Forks
3.5k
Avg merge
2d 11h
Merged PRs (30d)
132

Description

### Feature Request / Improvement

Athena throws the following error for CTAS statements that use Timestamp (with time zone) columns:

> Unsupported Hive type: timestamp(6) with time zone

See the below list of supported types:
https://docs.aws.amazon.com/athena/latest/ug/querying-iceberg-supported-data-types.html

> Only Iceberg timestamp (without time zone) is supported for Athena Iceberg DDL statements like CREATE TABLE, but all timestamp types can be queried through Athena.

We suggest adding a config option that allows for timestamp columns to be created without timezone.

```
private static final String TABLES_SCHEMA_LOGICAL_TIMESTAMP_WITHOUT_ZONE_PROP =
"iceberg.tables.schema-logical-timestamp-without-zone";
```

Currently in SchemaUtils.java:

```
case INT64:
if (Timestamp.LOGICAL_NAME.equals(valueSchema.name())) {
return TimestampType.withZone();
}
return LongType.get();

```

One could change it to something like this:

```
case INT64:
if (Timestamp.LOGICAL_NAME.equals(valueSchema.name())) {
if (config.schemaLogicalTimestampWithoutZone()) {
return TimestampType.withoutZone();
} else {
return TimestampType.withZone();
}
}
return LongType.get();
```

### Query engine

None

### Willingness to contribute

- [x] I can contribute this improvement/feature independently
- [x] I would be willing to contribute this improvement/feature with guidance from the Iceberg community
- [ ] I cannot contribute this improvement/feature at this time

Contributor guide

Open the contributing guide

Research direction

Start with the INT64 and Timestamp.LOGICAL_NAME handling in SchemaUtils.java, then trace the Connect configuration behind the proposed schema-logical-timestamp-without-zone property. Check the relevant existing configuration and test entry points before implementing the option. Done means the option selects TimestampType.withoutZone() for logical INT64 timestamps while the default remains withZone(), with coverage for both settings.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.