dbt-labs / dbt-labs/dbt-adapters

[CT-2343] [Bug] snapshot with custom datatypes breaks when source table is regenerated

Open
#678 9 comments 0 reactions 0 assignees View on GitHub
feature:snapshots pkg:dbt-postgres type:bug
Dominant language
Python
Stars
233
Forks
362
Avg merge
3d 22h
Merged PRs (30d)
9

Description

### Is this a new bug in dbt-core?

- [X] I believe this is a new bug in dbt-core
- [X] I have searched the existing issues, and I could not find an existing issue for this bug

### Current Behavior

I've set-up a snapshot table in a postgres database, and it has a column named `annotation_type` that uses a custom enum datatype. This has worked without problems for weeks, but after the table had to be truncated and restored due to a faulty replica setup we encountered the following error when trying t:
```
12:22:53 Database Error in snapshot annotations_image_snapshot (snapshots/annotations_image_snapshot.sql)
12:22:53 syntax error at or near "USER"
12:22:53 LINE 3: ...ions_image_snapshot" add column "annotation_type" USER-DEFIN...
```
I cannot find the sql file that the error is in reference to.
the snapshot table was also suddenly lacking the `annotation_type` column, despite snapshots supposedly never deleting columns?
We are at a development stage where we can still afford to do a full restart of the snapshot, but we'd rather not encounter this in the future.
Our hypothesis is that the custom type is somehow not matched appropriately following regeneration?

### Expected Behavior

I would expect snapshots to be robust to no-ops like this, and at least to never incur data loss.
If custom types are unsupported/problematic, it'd be nice to be warned.

### Steps To Reproduce

Not easily reproduced, but most likely course:
1. build source table with custom enum type in postgres
2. take snapshot
3. truncate table, then rebuild
4. take snapshot

### Relevant log output

_No response_

### Environment

_No response_

### Which database adapter are you using with dbt?

postgres

### Additional Context

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.