dbt-labs / dbt-labs/dbt

[CT-1747] Enable flagging nodes/configs for deferred validation

Open
#6,511 1 comment 3 reactions 0 assignees View on GitHub
engine:v1 type:tech-debt
Dominant language
Rust
Stars
13.8k
Forks
2.6k
Avg merge
21h 31m
Merged PRs (30d)
56

Description

Currently we have an issue with snapshot config, where the validation requires that certain config attributes be set. Since all node configs are validated twice -- once at model parsing time and once at schema parsing time (if a schema config exists) -- if somebody attempts to set the snapshot config in a schema file it will fail on the model parsing step.

In a more recent initiative we have config that might be set in both/either model and schema file and are attempting to validate that all of the config is in place. If the feature is turned on in model config and the other config settings are in the schema file, the complete validation would fail on model parsing.

What we need to handle situations like this is to be able to flag certain validation situations to happen after all of the schema parsing has been done. For example if a snapshot node is constructed from a sql file but doesn't contain some of the required config, set a flag. If a schema file runs that validation and it succeeds, we can then turn the flag off. But if we reach the end of schema file parsing and the flag is still set, re-run validation and issue an error.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.