dbt-labs / dbt-labs/dbt-adapters

[Bug] Snapshot with `external_location` config making empty table since 2nd run

Open
#450 0 comments 0 reactions 0 assignees View on GitHub
feature:snapshots pkg:dbt-athena type:bug
Dominant language
Python
Stars
233
Forks
362
Avg merge
3d 22h
Merged PRs (30d)
9

Description

When I make snapshot table
everything goes fine if there is no `external_location` setted in config. (but with `s3_data_naming='unique'`)

But, when I give `external_location` option on snapshot config
then table becoming empty since 2nd try .

I traced query logs and got a clue Imo.

when snapshot since 2nd run it gives 3 query in sequence below.

1. Create `{table_name}__dbt_tmp` table
- s3 location : at `s3_data_dir` with `s3_data_naming` convention (not external location)
2. Create `{table_name}__dbt_tmp_snapshot` table
- s3 location : not `s3_data_dir` but `external_location`
3. Create `{table_name}` table
- s3 location : at `external_location` same location as step 2

Actually there are no errors while executing those 3 queries. I guess after step3 finished the dbt removes s3 directory made at step2 which is same as of step3s.
I think make external location to `s3_data_dir`(like step1) at step 2 gonna solve this problem.
please check this symptom out.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.