dbt-labs / dbt-labs/dbt

[1.x Bug] state:modified selects all models under a parent block when a null-valued key in dbt_project.yml is changed

Open
#16,290 0 comments 0 reactions 0 assignees View on GitHub
bug engine:v1 status:triage triage type:bug
Dominant language
Rust
Stars
13.8k
Forks
2.6k
Avg merge
21h 31m
Merged PRs (30d)
56

Description

### Is this a new bug in dbt-core?

- [x] I believe this is a new bug in dbt-core
- [x] I have searched the existing issues, and I could not find an existing issue for this bug

### Current Behavior

If a key in the `dbt_project.yml` model tree has a **null value** (declared with
no children and no configs), then **any** change to that key causes
`state:modified` to select every model beneath the **parent** block, including
models in sibling subfolders that share no config with the edited key.

Both populating the null key and deleting it entirely trigger this. Adding a
child to an already-*populated* key does not.

All of it lands in `state:modified.configs`; `state:modified.body` and
`.macros` are 0.

### Expected Behavior

Per [Caveats to state comparison](https://docs.getdbt.com/reference/node-selection/state-comparison-caveats),
changes to `tags` and `meta` "don't count as modifications and will not trigger
`state:modified` ... This is intentional behavior." Adding a `+tags` entry
should select nothing. It certainly should not select unrelated models in
sibling folders.

### Steps To Reproduce

1. Create a project where a key in the model tree is null-valued:

```yaml
# dbt_project.yml
name: repro
profile: repro
version: "1.0"
config-version: 2

models:
repro:
parent:
alpha:
+tags: ["alpha"]
beta: # <- NULL key, no value
gamma:
+tags: ["gamma"]
```

2. Add four trivial models, each `select 1 as id`:
`models/parent/alpha/a1.sql`, `models/parent/alpha/a2.sql`,
`models/parent/beta/b1.sql`, `models/parent/gamma/g1.sql`

3. Save baseline state:

```bash
dbt parse && mkdir -p state && cp target/manifest.json state/
```

4. Add a tag under the null key:

```yaml
beta:
b1:
+tags: ["added"]
```

5. Select:

```bash
dbt ls --select state:modified --state state/ --resource-type model
```

**Expected:** no nodes selected.
**Actual:** all four models selected.

```
repro.parent.alpha.a1
repro.parent.alpha.a2
repro.parent.beta.b1
repro.parent.gamma.g1
```

`alpha` and `gamma` were not edited and share no config with `beta`.

### Results matrix

| edit | selected |
|---|---|
| none (baseline vs itself) | 0 |
| child added under populated `alpha:` | 0 |
| child added under null `beta:` | **4 (all)** |
| null `beta:` line deleted | **4 (all)** |

### Relevant log output

```shell
$ dbt ls --select state:modified --state state/ --resource-type model
Found 4 models, 500 macros
repro.parent.alpha.a1
repro.parent.alpha.a2
repro.parent.beta.b1
repro.parent.gamma.g1
```

### Environment

```markdown
- OS: macOS
- Python: 3.13
- dbt: 1.12.4
- adapter: dbt-duckdb 1.11.0
```

### Which database adapter are you using with dbt?

bigquery, other (mention it in "Additional Context")

### Additional Context

other (duckdb; also observed on BigQuery)

Not flag-dependent: reproduces identically with
`state_modified_compare_more_unrendered_values` set to `True`, `False`, and
absent. This rules out [#16133](https://github.com/dbt-labs/dbt-core/issues/16133),
whose only code path (`upgrade_unrendered_config`) is flag-gated, requires a
`--state` manifest version mismatch, and reads `node["raw_code"]` rather than
`dbt_project.yml`.

No deprecation warning required either: `CustomKeyInConfigDeprecation` does not
fire in this minimal project (checked with `dbt parse --no-partial-parse`), yet
it still reproduces.

I originally hit this issue in a production project with several hundred models: one small config edit under
a null key caused `state:modified+` to select an order of magnitude more models
than were actually affected, timing out a CI job with a 1-hour limit.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.