[1.x Bug] Partial parsing dropping YAML snapshot changes when coupled with its upstream source changes
- Dominant language
- Rust
- Stars
- 13.8k
- Forks
- 2.6k
- Avg merge
- 21h 31m
- Merged PRs (30d)
- 56
Description
### Is this a new bug in dbt-core?
- [x] I believe this is a new bug in dbt-core
- [x] I have searched the existing issues, and I could not find an existing issue for this bug
### Current Behavior
When a YAML-based snapshot and its upstream dependency change in the same partial parse, the snapshot's own changes are silently dropped - the manifest keeps the pre-change snapshot node. A full parse always works correctly and produces the expected outcome.
This behaviour appears to be order-dependent: it only triggers when the source's schema file is processed before the snapshot's in reverse-sorted file-id order — e.g. both under `models/`, with `models/sources.yml` sorting ahead of the snapshot's file.
**Related but distinct issues**
- #10907 added YAML-snapshot partial-parse handling (`delete_yaml_snapshot`) for a change to the snapshot — this path still works and is unaffected; the linked issue does not cover the upstream-co-change trigger.
- #11444 is the same family (stale YAML-snapshot node under partial parse) but a different trigger
### Expected Behavior
Under partial parse the snapshot node reflects the edited YAML, identical to a full parse.
### Steps To Reproduce
Minimal dbt project:
`models/sources.yml`
```yaml
version: 2
sources:
- name: my_source
schema: my_schema
tables: [{name: my_table}]
```
`models/a_snap.yml` *(under `models/`, named so `sources.yml` reverse-sorts ahead of it)*
```yaml
snapshots:
- name: my_snapshot
relation: "source('my_source', 'my_table')"
config: {strategy: check, unique_key: id, check_cols: all, meta: {owner: team_a}}
```
1. `dbt parse` → builds `target/partial_parse.msgpack`.
2. Changing **both**:
- add `description: changed` to `my_source` **and**
- set the snapshot's `meta.owner` → `team_b`
4. `dbt parse` (partial) → `manifest.json` snapshot `config.meta.owner` still points to **`team_a`** -> bug ❌
5. `rm target/partial_parse.msgpack && dbt parse` → **`team_b`** - works as expected ✅
### Relevant log output
```shell
```
### Environment
```markdown
- OS: macOS
- Python: 3.13
- dbt: 1.14.0a1 (tip of 1.latest); also observed on 1.11.x. Likely present since YAML `relation:` snapshots were introduced (1.9)
```
The bug is adapter-independent (parse-only) but originally identified in BigQuery.
### Which database adapter are you using with dbt?
bigquery
### Additional Context
Root cause in `core/dbt/parser/partial.py`:
- `remove_mssat_file` returns early on `not isinstance(source_file, SourceFile)`, so it **no-ops** on a snapshot's `SchemaSourceFile` (YAML-snapshot ids live in `SchemaSourceFile.snapshots`, not `SourceFile.nodes`).
- when the source change schedules the snapshot for re-parse, `schedule_nodes_for_parsing` overwrites `saved_files[file_id] = deepcopy(new_files[file_id])`.
- the snapshot's own `change_schema_file` then diffs **new-vs-new**, so `get_diff_for("snapshots")` is empty and `delete_yaml_snapshot` never fires → the stale node maintained
Contributor guide
Assessment
This issue has not been assessed yet.