dbt-labs / dbt-labs/dbt-adapters
[Bug] Snapshot with `external_location` config making empty table since 2nd run
- Dominant language
- Python
- Stars
- 233
- Forks
- 362
- Avg merge
- 3d 22h
- Merged PRs (30d)
- 9
Description
When I make snapshot table
everything goes fine if there is no `external_location` setted in config. (but with `s3_data_naming='unique'`)
But, when I give `external_location` option on snapshot config
then table becoming empty since 2nd try .
I traced query logs and got a clue Imo.
when snapshot since 2nd run it gives 3 query in sequence below.
1. Create `{table_name}__dbt_tmp` table
- s3 location : at `s3_data_dir` with `s3_data_naming` convention (not external location)
2. Create `{table_name}__dbt_tmp_snapshot` table
- s3 location : not `s3_data_dir` but `external_location`
3. Create `{table_name}` table
- s3 location : at `external_location` same location as step 2
Actually there are no errors while executing those 3 queries. I guess after step3 finished the dbt removes s3 directory made at step2 which is same as of step3s.
I think make external location to `s3_data_dir`(like step1) at step 2 gonna solve this problem.
please check this symptom out.
Contributor guide
Assessment
This issue has not been assessed yet.