dbt-labs / dbt-labs/dbt

[1.x Bug] data contracts expect the wrong timestamp datatype with athena connector

Open
#15,731 0 comments 1 reaction 0 assignees View on GitHub
area:adapters bug engine:v1 status:triage triage type:bug
Dominant language
Rust
Stars
13.8k
Forks
2.6k
Avg merge
21h 31m
Merged PRs (30d)
56

Description

### Is this a new bug in dbt-core?

- [x] I believe this is a new bug in dbt-core
- [x] I have searched the existing issues, and I could not find an existing issue for this bug

### Current Behavior

### Background

- Project uses the Athena connector with data_contracts enabled
- The source glue table has a timestamp column of type `timestamp(6) with time zone`. No transformations are applied to the column, so the resulting table has the same data type (when data_contracts are disabled)
- The profiles.yaml defines the expected data type as `timestamp with time zone`

### Current Behavior
- When executing `dbt run`, data contract fails because the definition_type is `TIMESTAMP` instead of expected `TIMESTAMP WITH TIME ZONE` (see attached screenshot)

Image

### Expected Behavior

- DBT run should succeed, creating the target table with a column `some_ts_with_timezone`
- The athena query `select distinct typeof(some_ts_with_timezone) from ` should return `timestamp(6) with time zone`

### Steps To Reproduce

- create a glue table, `my_source_db.my_source_table`, with a column, `my_timestamp_column`, of type `timestamp(6) with time zone`
- create a dbt project using the athena connector with data contracts enforced
- create a dbt model with sql that contains the following:
- a CTE, `my_cte`, which selects `my_timestamp_column` from `my_source_db.my_source_table` (the CTE is required to replicate!)
- a final query which selects `my_timestamp_column` from `my_cte`
- create a properties.yaml for the model and specify the data_type for `my_timestamp_column` as `timestamp with time zone`
- run the command `dbt run --select `, which will encounter a data contract failure

**screenshot of properties.yaml:**

Image

**screenshot of sql:**

Image

**screenshot of failure:**

Image

### Relevant log output

```shell
17:39:22 Completed with 1 error, 0 partial successes, and 0 warnings:
17:39:22
17:39:22 Compilation Error in model replicate_contracts_bug (models/replicate_contracts_bug/replicate_contracts_bug.sql/replicate_contracts_bug.sql)
This model has an enforced contract that failed.
Please ensure the name, data_type, and number of columns in your contract match the columns in your model's definition.

| column_name | definition_type | contract_type | mismatch_reason |
| ------------------- | --------------- | ------------------------ | ------------------ |
| my_timestamp_column | TIMESTAMP | TIMESTAMP WITH TIME ZONE | data type mismatch |


> in macro assert_columns_equivalent (macros/relations/column/columns_spec_ddl.sql)
> called by macro default__get_assert_columns_equivalent (macros/relations/column/columns_spec_ddl.sql)
> called by macro get_assert_columns_equivalent (macros/relations/column/columns_spec_ddl.sql)
> called by macro athena__create_table_as (macros/materializations/models/table/create_table_as.sql)
> called by macro create_table_as (macros/materializations/models/table/create_table_as.sql)
> called by macro safe_create_table_as (macros/materializations/models/table/create_table_as.sql)
> called by macro materialization_table_athena (macros/materializations/models/table/table.sql)
> called by model replicate_contracts_bug (models/replicate_contracts_bug/replicate_contracts_bug.sql/replicate_contracts_bug.sql)
17:39:22
```

### Environment

```markdown
- OS:maxOS 26.5
- Python: 3.12.9
- dbt: 1.9.4
- Plugins:
- athena: 1.9.3
```

### Which database adapter are you using with dbt?

other (mention it in "Additional Context")

### Additional Context

Our project uses the Athena connector (1.9.3). This issue only occurs when selecting from an intermediary CTE, not when doing a straight select from the table. If data_contracts are suppressed, the resulting target column has the expected data type `timestamp(6) with time zone`, not `timestamp(6)` as indicated by the contract_type output.

A workaround we have been using: in the sql, explicitly cast the source column as `timestamp(6) with time zone`. This casting is redundant (in theory), but it does prevent the data contract error.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.