[Enhancement] Support SQL expressions within `check_cols` for snapshots
- Dominant language
- Rust
- Stars
- 13.8k
- Forks
- 2.6k
- Avg merge
- 21h 31m
- Merged PRs (30d)
- 56
Description
### Is this a new bug in dbt-core?
- [x] I believe this is a new bug in dbt-core
- [x] I have searched the existing issues, and I could not find an existing issue for this bug
### Current Behavior
Hello,
While using dbt snapshot, with the following .yaml file:
```
snapshots:
- name: snapshot_test
relation: source('david_playground', 'table_with_nested_field')
config:
strategy: check
unique_key: id
check_cols:
- repeated_string
dbt_valid_to_current: "TIMESTAMP(DATE('9999-12-31'))"
hard_deletes: invalidate
```
It raises an error, because ARRAY (repeated field) are not comparable. This behaviour is good.
Then, I try to use `TO_JSON_STRING(repeated_string)` instead of `repeated_string` to have a proper SQL comparison.
Instead of trying to make the comparison, DBT release the following SQL for the update statement part:
```
updates as (
select
'update' as dbt_change_type,
source_data.*,
snapshotted_data.dbt_scd_id
from updates_source_data as source_data
join snapshotted_data
on
snapshotted_data.dbt_unique_key = source_data.dbt_unique_key
where ((TRUE))
)
```
Instead of trying to make the condition with TO_JSON_STRING(repeated_string), it just put TRUE.
No error is raised, and the snapshot will consider all rows of my source like updates, even if there is no update.
This is not expected in a snapshot in which I want to track only changes.
### Expected Behavior
Solution 1 : generate the condition with `TO_JSON_STRING(snapshotted_data.repeated_string) != TO_JSON_STRING(source_data.repeated_string)`
or
Solution 2 : raise an error to say SQL Function are not usable here
### Steps To Reproduce
1. In Bigquery, create a source table with id and repeated value
2. Run `dbt snapshot`
3. It's not working with the repeated value
4. Try to use `TO_JSON_STRING(your_repeated_field)` in `check_cols` of the yaml fille
5. Run `dbt snapshot` again
6. It's working incorrectly
### Relevant log output
```shell
```
### Environment
```markdown
- OS: MacOS
- Python: 3.11
- dbt: 1.9.4
```
### Which database adapter are you using with dbt?
bigquery
### Additional Context
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.