dbt-labs / dbt-labs/dbt-adapters

[Feature] simplify codepath for `load_cached_relation` jinja macro

Open
#189 0 comments 0 reactions 0 assignees View on GitHub
type:enhancement
Dominant language
Python
Stars
233
Forks
362
Avg merge
3d 22h
Merged PRs (30d)
9

Description

### Is this your first time submitting a feature request?

- [X] I have read the [expectations for open source contributors](https://docs.getdbt.com/docs/contributing/oss-expectations)
- [X] I have searched the existing issues, and I could not find an existing issue for this feature
- [X] I am requesting a straightforward extension of existing dbt-adapter functionality, rather than a Big Idea better suited to a discussion

### Describe the feature

an adapter maintainer (@prdpsvs) was trying to debug a scenario in which the first line of all the `global_project` materializations, `load_cached_relation` was returning `None`. The actual offending macros was 6 function calls away.

While there are good reasons things are this way, it has a spaghetti smell to me. Even if it can't be simplified, there's opportunity to better document this code path

### usage of `load_cached_relation`

https://github.com/dbt-labs/dbt-adapters/blob/6d097eead454faa49601d4426c87ec3213068f92/dbt/include/global_project/macros/materializations/models/table.sql#L3

### `load_cached_relation()` macro

effectively a wrapper around `adapter.get_relation()`

https://github.com/dbt-labs/dbt-adapters/blob/6d097eead454faa49601d4426c87ec3213068f92/dbt/include/global_project/macros/adapters/relation.sql#L67-L74

### `BaseAdapter.get_relation()`

calls `adapter.list_relations` and compares to the provided 3-part name does some matching

https://github.com/dbt-labs/dbt-adapters/blob/6d097eead454faa49601d4426c87ec3213068f92/dbt/adapters/base/impl.py#L859-L876

### `BaseAdapter.list_relations()`

if there isn’t already a cache, calls `BaseAdapter.list_relations_without_caching()`

https://github.com/dbt-labs/dbt-adapters/blob/6d097eead454faa49601d4426c87ec3213068f92/dbt/adapters/base/impl.py#L787-L821

### `BaseAdapter.list_relations_without_caching()`

invokes the jinja macro `list_relations_without_caching()` and does some comparison. The macro name is defined as a constant in the SQLAdapter.

https://github.com/dbt-labs/dbt-adapters/blob/6d097eead454faa49601d4426c87ec3213068f92/dbt/adapters/sql/impl.py#L13

https://github.com/dbt-labs/dbt-adapters/blob/6d097eead454faa49601d4426c87ec3213068f92/dbt/adapters/sql/impl.py#L183-L206

### `list_relations_without_caching` macro

should dispatch to the adapter specific version of the macro because `default__list_relations_without_caching` is set to raise a NotImplemented exception.

https://github.com/dbt-labs/dbt-adapters/blob/6d097eead454faa49601d4426c87ec3213068f92/dbt/include/global_project/macros/adapters/metadata.sql#L71-L78

### `fabric__list_relations_without_caching` macro

finally we've arrived at the true adapter zone code to be modified

https://github.com/microsoft/dbt-fabric/blob/adf9f8d6557e41eaae8ddacc14a08c057d67a687/dbt/include/fabric/macros/adapters/metadata.sql#L28-L50

```sql
{% macro fabric__list_relations_without_caching(schema_relation) -%}
{% call statement('list_relations_without_caching', fetch_result=True) -%}
USE [{{ schema_relation.database }}];
with base as (
select
DB_NAME() as [database],
t.name as [name],
SCHEMA_NAME(t.schema_id) as [schema],
'table' as table_type
from sys.tables as t {{ information_schema_hints() }}
union all
select
DB_NAME() as [database],
v.name as [name],
SCHEMA_NAME(v.schema_id) as [schema],
'view' as table_type
from sys.views as v {{ information_schema_hints() }}
)
select * from base
where [schema] like '{{ schema_relation.schema }}'
{% endcall %}
{{ return(load_result('list_relations_without_caching').table) }}
{% endmacro %}
```

### Describe alternatives you've considered

_No response_

### Who will this benefit?

- adapter maintainers
- end users who are trying to debug

### Are you interested in contributing this feature?

_No response_

### Anything else?

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.