[CT-2230] [Bug] snapshots not working properly with hive when specified column is not lowercase
- Dominant language
- Rust
- Stars
- 13.8k
- Forks
- 2.6k
- Avg merge
- 21h 31m
- Merged PRs (30d)
- 56
Description
### Is this a new bug in dbt-core?
- [X] I believe this is a new bug in dbt-core
- [X] I have searched the existing issues, and I could not find an existing issue for this bug
### Current Behavior
Hive metastore stores all column names as lowercase, even if they're explicitly created with uppercase names.
When you write a snapshot config that has the `check` strategy, and specify columns with uppercase column names, `dbt snapshot` invalidates all old records and copies all new records, even if there are no changes at all.
### Expected Behavior
`dbt snapshot` throws an error if the column does not exist.
For this specific case with hive metastore: if `UPPERCASE_COLUMN` gets stored as `uppercase_column`, but all the snapshot config is still written with `UPPERCASE_COLUMN`, `dbt snapshot` should complain that `UPPERCASE_COLUMN` does not exist.
### Steps To Reproduce
1. Have an adapter that uses hive metastore
2. Create a table with an uppercase column, e.g. `UPPERCASE_COLUMN`
3. `select *` from your table, and observe that the column name is now lowercase, as hive metastore has lowercased it
4. Create a snapshot from this table with the `check` strategy and specifiy the `UPPERCASE_COLUMN` as a column to check
5. Create another snapshot from this table (without changing anything, just run `dbt snapshot` again)
6. Select * from your snapshot and observe that the data is in there twice, with half of the records invalidated
### Relevant log output
_No response_
### Environment
```markdown
- OS: Ubuntu 22.04
- Python: 3.10.6
- dbt: 1.4.1
```
### Which database adapter are you using with dbt?
other (mention it in "Additional Context")
### Additional Context
We use the trino adapter, with a hive metastore and underlying iceberg.
Contributor guide
Assessment
This issue has not been assessed yet.