dbt-labs / dbt-labs/dbt

[FEAT] metadata via information_schema views

Open
#13,196 1 comment 0 reactions 3 assignees Claimed by @aahel View on GitHub
area:engine engine:v2 status:triage type:feature
Dominant language
Rust
Stars
13.8k
Forks
2.6k
Avg merge
21h 31m
Merged PRs (30d)
56

Description

**Is your feature request related to a problem? Please describe.**
One feature of sdf that I loved was the [sdf information schema views](https://docs.sdf.com/sdf-information-schema). This felt (and still feels) inevitable and correct to me because it mirrors the mechanism that databases use to expose metadata.

SQL to analyse your SQL >> python/jq to analyse your SQL
Using a familiar, existing generic skill rather than learning a new, custom skill.

Would provide an alternative syntax to dbt cli.

**Is your feature request related to a particular component of the dbt Fusion Engine**
- [X] CLI Features
- [ ] Adapter or Database Driver Features
- [X] SQL Understanding Features
- [ ] VS Code extension

**Describe the solution you'd like**
To start, copy/paste of the relevant subset of the sdf implementation would be welcome.

I would like to see this embraced and extended by making tests (of both kinds) included in the information_schema views.

Taking this to the logical extreme, what if, the entire manifest.json file was queryable via a SQL API as information schema tables?

I can imagine enabling scenarios:
- dbt models about dbt models (ala sdf checks) and
- dbt tests on dbt models about dbt models.
- simplify dbt-project-evaluator
- simplified, first party version of coverage analysis ala github.com/slidiapp/dbt-coverage/blob/main/readme. Faster is better. More teams would use it.

**Describe alternatives you've considered**

- duckdb over manifest.json.
- Load manifest.json into a sqlite/Apache Fusion database.

Either of these might be viable but adoption would be significantly less if it is not a an official first party feature. A custom mod would be significantly less analytics engineer friendly than it could be or at least a common objection from colleagues who perceive it as adding complexing and risk.

**Additional context**
Add any other context or screenshots about the feature request here.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.