Should column level `meta` modifications be considered `state:modified`?
- Dominant language
- Rust
- Stars
- 13.8k
- Forks
- 2.6k
- Avg merge
- 21h 31m
- Merged PRs (30d)
- 56
Description
### Is this a new bug in dbt-core?
- [X] I believe this is a new bug in dbt-core
- [X] I have searched the existing issues, and I could not find an existing issue for this bug
### Current Behavior
As title, changes to column level `meta` configs aren't considered `state:modified`.
### Expected Behavior
Changes to column level `meta` configs should be considered `state:modified`.
### Steps To Reproduce
1. Setup dbt project.
```yml
# dbt_project.yml
name: my_dbt_project
profile: all
config-version: 2
version: "1.0.0"
models:
my_dbt_project:
+materialized: table
```
```yml
# models/schema.yml
version: 2
models:
- name: foo
config:
meta:
k: v
columns:
- name: id
meta:
k: v
- name: bar
config:
meta:
k: v
columns:
- name: id
meta:
k: v
```
```sql
-- models/foo.sql
select 1 id
-- models/bar.sql
select 1 id
```
2. Initial build and state storing.
```sh
$ dbt build && mv target target_old
00:00:49 Running with dbt=1.7.14
00:00:49 Registered adapter: postgres=1.7.14
00:00:49 Found 2 models, 0 sources, 0 exposures, 0 metrics, 402 macros, 0 groups, 0 semantic models
00:00:49
00:00:49 Concurrency: 4 threads (target='dev')
00:00:49
00:00:49 1 of 2 START sql table model public.bar ........................................ [RUN]
00:00:49 2 of 2 START sql table model public.foo ........................................ [RUN]
00:00:49 1 of 2 OK created sql table model public.bar ................................... [SELECT 1 in 0.08s]
00:00:49 2 of 2 OK created sql table model public.foo ................................... [SELECT 1 in 0.08s]
00:00:49
00:00:49 Finished running 2 table models in 0 hours 0 minutes and 0.17 seconds (0.17s).
00:00:49
00:00:49 Completed successfully
00:00:49
00:00:49 Done. PASS=2 WARN=0 ERROR=0 SKIP=0 TOTAL=2
```
3. Modify `schema.yml` file (a model level change on `foo`, and a column level change on `bar`):
```yml
# models/schema.yml
version: 2
models:
- name: foo
config:
meta:
k: model_level_change
columns:
- name: id
meta:
k: v
- name: bar
config:
meta:
k: v
columns:
- name: id
meta:
k: column_level_change
```
4. Deferred build:
```sh
$ dbt build -s state:modified --defer --state target_old
00:02:48 Running with dbt=1.7.14
00:02:48 Registered adapter: postgres=1.7.14
00:02:48 Unable to do partial parsing because saved manifest not found. Starting full parse.
00:02:48 Found 2 models, 0 sources, 0 exposures, 0 metrics, 402 macros, 0 groups, 0 semantic models
00:02:48
00:02:48 Concurrency: 4 threads (target='dev')
00:02:48
00:02:48 1 of 1 START sql table model public.foo ........................................ [RUN]
00:02:48 1 of 1 OK created sql table model public.foo ................................... [SELECT 1 in 0.06s]
00:02:48
00:02:48 Finished running 1 table model in 0 hours 0 minutes and 0.15 seconds (0.15s).
00:02:48
00:02:48 Completed successfully
00:02:48
00:02:48 Done. PASS=1 WARN=0 ERROR=0 SKIP=0 TOTAL=1
```
> `foo` is built but not `bar`.
### Relevant log output
```shell
Shared above.
```
### Environment
```markdown
- OS: macOS
- Python: 3.11.9
- dbt:
Core:
- installed: 1.7.14
- latest: 1.8.0 - Update available!
Plugins:
- postgres: 1.7.14 - Update available!
```
### Which database adapter are you using with dbt?
postgres
### Additional Context
Not 100% sure if this has ever worked - could be FR like https://github.com/dbt-labs/dbt-core/issues/10020
Contributor guide
Assessment
This issue has not been assessed yet.