dbt-labs / dbt-labs/dbt-adapters

DBT custom snapshot materialization not taking the config `snapshot_meta_column_names`

Open
#1,375 7 comments 0 reactions 0 assignees View on GitHub
feature:snapshots triage:product type:bug
Dominant language
Python
Stars
233
Forks
362
Avg merge
3d 22h
Merged PRs (30d)
9

Description

### Is this a new bug?

- [x] I believe this is a new bug
- [x] I have searched the existing issues, and I could not find an existing issue for this bug

### Which packages are affected?

- [x] dbt-adapters
- [ ] dbt-tests-adapter
- [ ] dbt-athena
- [ ] dbt-athena-community
- [ ] dbt-bigquery
- [ ] dbt-postgres
- [ ] dbt-redshift
- [x] dbt-snowflake
- [ ] dbt-spark

### Current Behavior

For the custom snapshot materialization, when you define custom values for snapshot_meta_column_names, the values are not considered, for example, when you call your custom materialization using :

`snapshot_meta_column_names={
'dbt_valid_from': 'valid_from_ts',
'dbt_valid_to': 'valid_to_ts',
'dbt_updated_at': 'row_updated_at',
'dbt_scd_id': 'surrogate_scd_id',
'dbt_is_current': 'is_current_flag'
}`

the materialization still running with default values and giving error :

`Snapshot target is missing configured columns (missing "dbt_scd_id", "dbt_valid_from", "dbt_valid_to"). See https://docs.getdbt.com/docs/build/snapshots#snapshot-meta-fields for more information.`

As per my analysis, the code for custom snapshot materialization not the default config snapshot_meta_column_names : https://github.com/dbt-labs/dbt-adapters/tree/65bc9440be852a8b4bb1e004c0926c32dcc35950/dbt-adapters/src/dbt/include/global_project/macros/materializations/snapshots

rather it is using snapshot_table_column_names.

### Expected Behavior

The custom snapshot materialization should not use snapshot_table_column_names and should use snapshot_meta_column_names

### Steps To Reproduce

Not working 👎 :

```sql
{{ config(
materialized='snapshot',
unique_key='customer_id',
strategy='check',
check_cols=['name','status'],
snapshot_meta_column_names={
'dbt_valid_from': 's_valid_from_ts',
'dbt_valid_to': 's_valid_to_ts',
'dbt_updated_at': 's_updated_at',
'dbt_scd_id': 's_scd_id',
'dbt_is_deleted': 's_deleted_flag'
}
) }}

select
CUSTOMER_ID, NAME, EMAIL, STATUS, TIER, CREATED_AT, UPDATED_AT
from {{ source('generic', 'customers') }}
```

Working version👍 :

```sql
{{ config(
materialized='snapshot',
unique_key='customer_id',
strategy='check',
check_cols=['name','status'],
snapshot_table_column_names={
'dbt_valid_from': 's_valid_from_ts',
'dbt_valid_to': 's_valid_to_ts',
'dbt_updated_at': 's_updated_at',
'dbt_scd_id': 's_scd_id',
'dbt_is_deleted': 's_deleted_flag'
}
) }}

select
CUSTOMER_ID, NAME, EMAIL, STATUS, TIER, CREATED_AT, UPDATED_AT
from {{ source('generic', 'customers') }}
```

### Relevant log output

```shell

```

### Environment

```markdown
- OS:
- Python:
- dbt-adapters:
- :
```

### Additional Context

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.