dbt-labs / dbt-labs/dbt

[CT-2429] `dbt show`: option to use already-materialized model instead of rerunning SQL

Open
#7,391 13 comments 21 reactions 0 assignees View on GitHub
engine:v1 paper_cut Refinement type:feature
Dominant language
Rust
Stars
13.8k
Forks
2.6k
Avg merge
21h 31m
Merged PRs (30d)
56

Description

As a dbt developer, especially on BigQuery, I want to preview my just-built table in the fastest & cheapest way possible.

---

When users run `dbt show --select my_model`, rather than recompiling and executing that model's SQL, give users the option to preview the already-materialized version of `my_model` in the data warehouse.

This goes beyond just `--inline "select * from {{ ref('that_model') }}"`, because it would also unlock adapter-specific optimizations.
- On all data platforms, this saves compute
- [On BigQuery, this saves $$$.](https://cloud.google.com/bigquery/docs/best-practices-costs#preview-data)

> DWHs may have optimizations that make it fast/cheap to show a sample of an existing (already-materialized) table, versus actually running the `select * from query`. Specifically on BigQuery - there's an API call you can make to show that sample, it's fast + free, whereas `select * from ` will charge you for scanning the entire table

Options:
- Config, `--from-materialized` (I hate this name)
- Different command: `dbt head`, to match `bq head`?

In the short term, it would be the responsibility of users to track logical/applied state, and do their own "cache invalidation" if they've edited the model's definition since they last built it. (In the longer term, we could get cleverer about this in stateful runtime environments.)

This would be functionally identical for views and ephemeral models.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.