[CT-705] Review `dbt_utils` "helper" methods: _is_relation + _is_ephemeral
- Dominant language
- Rust
- Stars
- 13.8k
- Forks
- 2.6k
- Avg merge
- 21h 31m
- Merged PRs (30d)
- 56
Description
Many `dbt_utils` macros need to access information about an underlying table, by running `adapter.get_columns_in_relation` or a custom query.
Those macros expect to be passed a `ref`, `source`, or other `Relation` object, and for it to represent an object that really exists in the database. To ensure that's the case, and raise a helpful error if it isn't, they first call "internal helper" macros, `_is_relation` and `_is_ephemeral`, to check that the passed argument (a) really is a Relation, (b) isn't ephemeral (= exists as a database object, assuming it's already been run).
Because this is Jinja, we can't just call `isinstance`, or classmethods. We have to check meta properties.
#### [`dbt_utils._is_relation`](https://github.com/dbt-labs/dbt-utils/blob/main/macros/cross_db_utils/_is_relation.sql)
```
{% macro _is_relation(obj, macro) %}
{%- if not (obj is mapping and obj.get('metadata', {}).get('type', '').endswith('Relation')) -%}
{%- do exceptions.raise_compiler_error("Macro " ~ macro ~ " expected a Relation but received the value: " ~ obj) -%}
{%- endif -%}
{% endmacro %}
```
#### [`dbt_utils._is_ephemeral`](https://github.com/dbt-labs/dbt-utils/blob/main/macros/cross_db_utils/_is_ephemeral.sql)
There is a node method, [is_ephemeral](https://github.com/dbt-labs/dbt-core/blob/75f3e8cb749c7b52d226844394ecb34c1e0070a4/core/dbt/contracts/graph/parsed.py#L138-L140), but nothing available on the actual `Relation` object returned by `ref`... except the [`is_cte`](https://github.com/dbt-labs/dbt-core/blob/75f3e8cb749c7b52d226844394ecb34c1e0070a4/core/dbt/adapters/base/relation.py#L318-L320) property:
```
{% macro _is_ephemeral(obj, macro) %}
{%- if obj.is_cte -%}
{% set ephemeral_prefix = api.Relation.add_ephemeral_prefix('') %}
{% if obj.name.startswith(ephemeral_prefix) %}
{% set model_name = obj.name[(ephemeral_prefix|length):] %}
{% else %}
{% set model_name = obj.name %}
{%- endif -%}
{% set error_message %}
The `{{ macro }}` macro cannot be used with ephemeral models, as it relies on the information schema.
`{{ model_name }}` is an ephemeral model. Consider making it a view or table instead.
{% endset %}
{%- do exceptions.raise_compiler_error(error_message) -%}
{%- endif -%}
{% endmacro %}
```
### Questions
- Is there a better way to write these? Are there better properties / classmethods we could expose on the `Relation` object, to make it accessible in the Jinja layer?
- Should these checks actually be happening within adapter methods like `adapter.get_columns_in_relation`?
Contributor guide
Assessment
This issue has not been assessed yet.