[Feature] Add ability to import/include YAML from other files
- Dominant language
- Rust
- Stars
- 13.8k
- Forks
- 2.6k
- Avg merge
- 21h 31m
- Merged PRs (30d)
- 56
Description
### Is this your first time submitting a feature request?
- [X] I have read the [expectations for open source contributors](https://docs.getdbt.com/docs/contributing/oss-expectations)
- [X] I have searched the existing issues, and I could not find an existing issue for this feature
- [X] I am requesting a straightforward extension of existing dbt functionality, rather than a Big Idea better suited to a discussion
### Describe the feature
I wonder if dbt would benefit from providing the ability to include YAML snippets from other files.
Home Assistant (a popular python tool, leveraging YAML for config), [provides a few additional YAML constructors](https://www.home-assistant.io/docs/configuration/splitting_configuration/) ([quick overview of constructors](https://matthewpburruss.com/post/yaml/#defining-pyyaml-constructors-going-from-yaml-to-python)).
The most interesting one would be `!include` which allows to define values of keys in YAML from other YAML files
The implementation of Home Assistant is [in this source file](https://github.com/home-assistant/core/blob/b629f5c143b4d2054b77913c550e10098825ed8b/homeassistant/util/yaml/loader.py#L507).
While we can define some logic in Jinja in some YAML files, we are limited today to defining a single entry at a time (e.g. a string or a bool). With this approach we can also store config for nested fields.
Based on my first observations, this technique wouldn't mess with the current dbt code as the YAML is rendered before we start any "real" dbt parsing.
### Describe alternatives you've considered
Copy pasting the same logic in YAML as we do today
### Who will this benefit?
- people who want to DRY their YAML files (e.g common tests for columns, common tags etc...)
- people who want to define the config of `dbt_project.yml` from different files, with different code owners in git
- people following [this issue](https://github.com/dbt-labs/dbt-core/issues/2955)
### Are you interested in contributing this feature?
Yes
### Anything else?
I got an early prototype working locally and can share some of the code if we want to implement this feature.
Contributor guide
Assessment
This issue has not been assessed yet.