dbt-labs / dbt-labs/dbt

[1.x Bug] Partial parsing dropping YAML snapshot changes when coupled with its upstream source changes

Open
#15,785 0 comments 0 reactions 0 assignees View on GitHub
area:engine engine:v1 status:triage type:bug
Dominant language
Rust
Stars
13.8k
Forks
2.6k
Avg merge
21h 31m
Merged PRs (30d)
56

Description

### Is this a new bug in dbt-core?

- [x] I believe this is a new bug in dbt-core
- [x] I have searched the existing issues, and I could not find an existing issue for this bug

### Current Behavior

When a YAML-based snapshot and its upstream dependency change in the same partial parse, the snapshot's own changes are silently dropped - the manifest keeps the pre-change snapshot node. A full parse always works correctly and produces the expected outcome.

This behaviour appears to be order-dependent: it only triggers when the source's schema file is processed before the snapshot's in reverse-sorted file-id order — e.g. both under `models/`, with `models/sources.yml` sorting ahead of the snapshot's file.

**Related but distinct issues**
- #10907 added YAML-snapshot partial-parse handling (`delete_yaml_snapshot`) for a change to the snapshot — this path still works and is unaffected; the linked issue does not cover the upstream-co-change trigger.
- #11444 is the same family (stale YAML-snapshot node under partial parse) but a different trigger

### Expected Behavior

Under partial parse the snapshot node reflects the edited YAML, identical to a full parse.

### Steps To Reproduce

Minimal dbt project:

`models/sources.yml`

```yaml
version: 2
sources:
- name: my_source
schema: my_schema
tables: [{name: my_table}]
```

`models/a_snap.yml` *(under `models/`, named so `sources.yml` reverse-sorts ahead of it)*

```yaml
snapshots:
- name: my_snapshot
relation: "source('my_source', 'my_table')"
config: {strategy: check, unique_key: id, check_cols: all, meta: {owner: team_a}}
```

1. `dbt parse` → builds `target/partial_parse.msgpack`.
2. Changing **both**:
- add `description: changed` to `my_source` **and**
- set the snapshot's `meta.owner` → `team_b`
4. `dbt parse` (partial) → `manifest.json` snapshot `config.meta.owner` still points to **`team_a`** -> bug ❌
5. `rm target/partial_parse.msgpack && dbt parse` → **`team_b`** - works as expected ✅

### Relevant log output

```shell

```

### Environment

```markdown
- OS: macOS
- Python: 3.13
- dbt: 1.14.0a1 (tip of 1.latest); also observed on 1.11.x. Likely present since YAML `relation:` snapshots were introduced (1.9)
```

The bug is adapter-independent (parse-only) but originally identified in BigQuery.

### Which database adapter are you using with dbt?

bigquery

### Additional Context

Root cause in `core/dbt/parser/partial.py`:

- `remove_mssat_file` returns early on `not isinstance(source_file, SourceFile)`, so it **no-ops** on a snapshot's `SchemaSourceFile` (YAML-snapshot ids live in `SchemaSourceFile.snapshots`, not `SourceFile.nodes`).
- when the source change schedules the snapshot for re-parse, `schedule_nodes_for_parsing` overwrites `saved_files[file_id] = deepcopy(new_files[file_id])`.
- the snapshot's own `change_schema_file` then diffs **new-vs-new**, so `get_diff_for("snapshots")` is empty and `delete_yaml_snapshot` never fires → the stale node maintained

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.