dbt-labs / dbt-labs/dbt

[v2 Bug] state:modified+ false positives for unchanged seeds/models with defer + environment comparison (dbt Cloud)

Open
#15,231 3 comments 0 reactions 1 assignee Claimed by @jeremyyeo View on GitHub
area:engine engine:v2 type:bug
Dominant language
Rust
Stars
13.8k
Forks
2.6k
Avg merge
21h 31m
Merged PRs (30d)
56

Description

### Is this a new bug in dbt v2.x compared to the latest version of dbt 1.x?

- [x] I believe this is a new bug in dbt v2.x
- [x] I have searched the existing issues and could not find a duplicate

### Current Behavior

On dbt Cloud (Fusion Stable, Snowflake), a deployment job configured with **Compare changes against: Environment** and **Compare changes against an environment (Deferral): Yes; Defer to PROD** selects and builds many unchanged nodes when running:

```bash
dbt seed --select state:modified
dbt build --select state:modified+ --exclude
```

Observed on each run:

- All project seeds are rebuilt (none changed in git)
- Large unrelated downstream subgraphs across staging, intermediate, and mart layers
- Job runtime goes from minutes to hours; we cancel before completion

Logs show `hydrating deferral` and `hydrating schema` at job start. Offline comparison of defer baseline vs current-run manifest shows **0 model checksum diffs**, **0 seed checksum diffs**, and **0 new nodes** — yet all 5 seeds and a large downstream graph still run. Runtime selection does not match what the manifests indicate should be modified.

This same job configuration worked correctly on **dbt Core 1.x** in dbt Cloud. The over-selection began after migrating the PROD environment to **Fusion Stable**; we did not change job commands or defer/comparison settings.

The defer baseline comes from a scheduled PROD job (not the slim CI job). We verified the baseline is a full project parse (~409 models), not a truncated manifest.

### Expected Behavior

With `state:modified` / `state:modified+` comparing current code against the PROD environment manifest:

- Unchanged seeds should **not** be selected
- Unchanged models should **not** be selected
- When manifests are identical, **nothing** should run (or only nodes with genuine checksum diffs)

### Steps To Reproduce

Steps in our dbt Cloud setup:

1. Configure a deployment job in a PROD environment with:
- **Compare changes against:** `Environment`
- **Compare changes against an environment (Deferral):** `Yes; Defer to PROD`
- **Commands:**
```bash
dbt seed --select state:modified
dbt build --select state:modified+ --exclude
dbt docs generate
```
2. Have a second scheduled job in the same environment succeed (updates the environment's stored manifest). Ours runs a package/tag subset; the stored manifest is still a full parse.
3. Trigger the slim CI job with no model or seed file changes (UI run or merge trigger).
4. Observe: all seeds selected; large unrelated downstream graph selected; runtime explodes.

**Regression:** This job ran correctly on dbt Core 1.x with the same commands and defer/comparison settings. A manifest from Jun 1 from the same slim CI job (`2026.5.28+225bfa3`, pre-Fusion) confirms normal behavior on GitHub push. After migrating to Fusion Stable, it began selecting all seeds and large unrelated downstream graphs — with no job config changes. Within Fusion, the job also completed in minutes with expected selection on `2.0.0-preview.188` (~Jun 9) before failing on `2.0.0-preview.186`.

**Offline verification:**

1. Download `manifest.json` from the slim CI run and from the defer baseline.
2. Compare each node's `checksum.checksum` value across both files.
3. Result (latest test): **0 model diffs, 0 seed diffs, 0 new nodes** — yet the slim CI job still selects all seeds + downstream.

**Suggested diagnostic** — add before the seed step:

```bash
dbt ls --select state:modified --output name
dbt ls --select state:modified --resource-type seed --output name
```

### Relevant log output

```shell
Sanitized excerpt from a failing slim CI run:

Running Fusion version: 2.0.0-preview.186
Running parsing
Running scheduling
Running hydrating deferral
Running hydrating schema
...
Started seed
Started seed
Started seed
Started seed
Started seed
...
Started model
Started model
Started model
...

All seeds start even though offline manifest comparison shows no seed checksum changes vs the defer baseline. Fusion Stable has reported `.186` or `.188` across our runs; over-selection happens on both.

> Can provide full log excerpts privately via dbt Support; omitting run/account/job IDs here.
```

### Environment

```markdown
To my knowledge, this reproduces only on dbt Cloud

| Setting | Value |
| --- | --- |
| **dbt distribution and version** | dbt Cloud, Fusion Stable — logs report `2.0.0-preview.186` or `2.0.0-preview.188` (cannot pin exact build in Cloud UI) |
| Platform | PROD deployment environment |
| Adapter | Snowflake |
| Project size | ~409 models, 5 seeds, ~608 tests |
| Compare changes against | `Environment` |
| Deferral | `Yes; Defer to PROD` |
| Commands | See Steps To Reproduce |
```

### Which database adapter are you using?

snowflake

### Is this a discrepancy vs. dbt 1.x?

- [x] Yes — this works in dbt 1.x but not in dbt v2.x

### Additional Context

**Sanitized manifest comparison** (full manifests not attached — contain Cloud account/job/run IDs and warehouse metadata):

| Comparison | Model checksum diffs | Seed checksum diffs | New nodes |
| --- | ---: | ---: | --- |
| Latest test run vs defer baseline (Jun 10) | 0 | 0 | 0 |
| Prior successful slim CI run vs failing run | 0 seeds; 1 model (`bin_data`) on older comparison | 0 | varies by merge |

**Fusion versions in those manifests:**

- Prior successful slim run: `2.0.0-preview.188`
- Failing runs + defer baseline: `2.0.0-preview.186`
- Prior distribution: dbt Core 1.x / pre-Fusion Cloud (`2026.5.28+225bfa3`, Jun 1; same job config; `state:modified` + defer behaved as expected)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.