[1.x Bug] Versioned models' `unrendered_config` is asymmetric — model-level block is rendered, version-level block is unrendered
- Dominant language
- Rust
- Stars
- 13.8k
- Forks
- 2.6k
- Avg merge
- 21h 31m
- Merged PRs (30d)
- 56
Description
### Is this a new bug in dbt-core?
- [x] I believe this is a new bug in dbt-core
- [x] I have searched the existing issues, and I could not find an existing issue for this bug
### Current Behavior
For a versioned model, `SchemaSourceFile.get_unrendered_config(yaml_key, name, version)` looks up only
the versioned key `f"{name}_v{version}"` and never falls back to (or merges with) the bare `name` entry
that holds the *model-level* `config:` block's unrendered form. As a result, when
`update_parsed_node_config` merges the unrendered override into `patch_config_dict` under
`state_modified_compare_more_unrendered_values`:
```python
if unrendered_patch_config := patch_file.get_unrendered_config(
schema_key, parsed_node.name, getattr(parsed_node, "version", None)
):
patch_config_dict = deep_merge(patch_config_dict, unrendered_patch_config)
```
only the **version-level** `config:` block is deep-merged in unrendered form. The **model-level**
`config:` block was already rendered earlier in the same function and is never replaced. The
`unrendered_config` ultimately recorded on a versioned node is therefore a mix: model-level keys are
rendered, version-level keys are unrendered.
Concretely, for:
```yaml
models:
- name: my_model
config:
persist_docs:
relation: "{{ var('persist_relation_docs', true) }}" # model level
latest_version: 1
versions:
- v: 1
defined_in: my_model_v1
config:
persist_docs:
columns: true # version level
```
the recorded `unrendered_config.persist_docs` comes out as `{"relation": true, "columns": true}` —
`relation` already rendered to a boolean (confirmed on a real `dbt parse` run), `columns` left as
authored (it had no Jinja to render in this example, so it isn't a good test of the version-level
path's own rendering — but the mechanism means it never gets the model-level treatment either way). If
`relation`'s Jinja depended on something environment-specific, the false-positive protection that
`state_modified_compare_more_unrendered_values` exists to provide is missing for the model-level block
of a versioned model, even though it's present for the identical config on an unversioned model.
**Separately, and less certain — please verify before treating as confirmed:** the same `deep_merge`
call prepends the version's *unrendered* values onto `patch_config_dict`, which for list-valued keys
(`tags`, `pre_hook`/`post_hook`) may already contain the *rendered* model-level values merged in
earlier in the same function. If so, a versioned model with both a model-level and version-level
`tags:` (or hook) would get `unrendered_config` list entries duplicated rather than merged once. We
have not captured a confirmed example of this specific duplication from a live run — flagging the code
path in case it's a real second symptom of the same root cause, not asserting the exact output shape.
### Expected Behavior
For a versioned model, `unrendered_config` should represent the fully merged, still-unrendered config
for that version — the model-level `config:` block deep-merged with the version-level `config:` block,
both left unrendered — analogous to how the *rendered* `config` is already assembled from both levels
via `deep_merge(target.config, unparsed_version.config)`. Currently only the rendered side gets that
full model-plus-version treatment; the unrendered side gets version-only.
### Steps To Reproduce
1. Author the `persist_docs` example above (or any model-level `config:` key that renders differently
under Jinja, e.g. gated on `target.name`).
2. `dbt parse`.
3. Inspect `target/manifest.json` for `model..my_model.v1`'s `unrendered_config`.
4. Observe that the model-level key's value is the **rendered** value, not the authored Jinja/raw
value, while the version-level key is left as authored.
### Relevant log output
```shell
(none — this is a data-shape issue in the manifest, not a runtime error)
```
### Environment
```markdown
- dbt: `1.latest` (1.14.0a1) as of this issue
- OS / Python: not relevant
```
### Which database adapter are you using with dbt?
N/A — parser/manifest-shape issue, not adapter-specific.
### Additional Context
Mechanism, cited against
[`1.latest` @ `f54a42c2e4`](https://github.com/dbt-labs/dbt-core/tree/f54a42c2e47dd5ff4bd9375851fa238cdcd4eb22):
- Storage keyed per version, holding only that version's raw `config:` block:
[`core/dbt/contracts/files.py:331-338`](https://github.com/dbt-labs/dbt-core/blob/f54a42c2e47dd5ff4bd9375851fa238cdcd4eb22/core/dbt/contracts/files.py#L331-L338)
(`add_unrendered_config`).
- Lookup, versioned-key-only, no fallback to the bare model name:
[`core/dbt/contracts/files.py:340-348`](https://github.com/dbt-labs/dbt-core/blob/f54a42c2e47dd5ff4bd9375851fa238cdcd4eb22/core/dbt/contracts/files.py#L340-L348)
(`get_unrendered_config`).
- Read side / merge into `patch_config_dict`:
[`core/dbt/parser/base.py:413-426`](https://github.com/dbt-labs/dbt-core/blob/f54a42c2e47dd5ff4bd9375851fa238cdcd4eb22/core/dbt/parser/base.py#L413-L426).
- For contrast, the rendered-config merge that already handles both levels together:
[`core/dbt/parser/schemas.py:1090`](https://github.com/dbt-labs/dbt-core/blob/f54a42c2e47dd5ff4bd9375851fa238cdcd4eb22/core/dbt/parser/schemas.py#L1090)
(`config=deep_merge(target.config, unparsed_version.config)`).
A `get_unrendered_config` that also fetches (and deep-merges with) the bare `name` entry when a
versioned lookup exists would fix the rendered/unrendered mismatch. If the list-duplication concern
above holds up, that fix would additionally need care to merge the two *unrendered* blocks together
once, rather than merging the version's unrendered block onto a `patch_config_dict` that may already
contain the model level merged in via the rendered path.
**Same root cause, non-model resource:** an unowned `TODO` in dbt Labs' Fusion codebase already
independently noticed the identical `deep_merge`-of-rendered-and-unrendered-configs shape for **seeds**
(which have no `versions:`, so it's the same underlying `add_unrendered_config`/`get_unrendered_config`
pattern, not the versioned-specific fallback gap above) — worth checking whether a fix for one should
consider the other.
**Cross-engine note:** dbt Labs' Fusion engine has decided ([dbt-labs/fs#13181](https://github.com/dbt-labs/fs/pull/13181))
**not** to replicate this asymmetry — for a
versioned model, Fusion emits `unrendered_config` fully unrendered at both the model and version level,
which is simpler and matches the (uncontroversial) rendered-vs-unrendered intent. This is believed to
be a strictly safer divergence (it can only cause a spurious Stage-1 diff that a later rendered-config
comparison then clears, never a missed detection), so we are not asking dbt Core to match Fusion here —
this issue is filed for dbt-core's own awareness/triage, not to request a specific engine-alignment
change the way issues 1 and 2 do.
Contributor guide
Assessment
This issue has not been assessed yet.