AltimateAI / AltimateAI/vscode-dbt-power-user

Could not load register_upstream_external_models in dbt-duckdb

Open
#1,381 6 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
JavaScript
Stars
585
Forks
121
Avg merge
5d 39m
Merged PRs (30d)
4

Description

### Expected behavior

I’m currently using dbt-duckdb and writing data to Minio. dbt run executes successfully and writes data to Minio, but I cannot execute Execute dbt SQL in the VS Code plugin.

When clicking "Execute dbt SQL", it is possible to query the raw model stored in S3 from the stg model.

And more, I’ve added the following configuration to my dbt_project.yml file:

```yaml
on-run-start:
- "{{ register_upstream_external_models() }}"
```

I found selected_resources is empty in macro register_upstream_external_models.
Therefore, the macro will do nothing.

### Actual behavior

When clicking "Execute dbt SQL", Query Results Show:

Runtime Error
Catalog Error: Table with name raw_events does not exist! Did you mean "pg_catalog.pg_views"? LINE 6: SELECT * FROM "memory"."main"."raw_events" ^

### Steps To Reproduce

file: s3://playground/raw/events.json
```json
{"datetime":"2024-07-01T00:03:48.072909+08:00","action":"login","user_id":1,"ip":"127.0.0.1"}
{"datetime":"2024-07-01T00:03:48.072909+08:00","action":"pay","user_id":1,"amount":2}
```

file: raw_events.sql
```sql
{{
config(
format='parquet',
incremental_strategy='insert_overwrite',
incremental=True,
location='s3://playground/warehouse/raw_events',
materialized='external',
options={"partition_by": "dw_date", "overwrite_or_ignore": 1}
)
}}

SELECT
datetime as dw_ts,
CAST(dw_ts AS DATE) AS dw_date,
* EXCLUDE (datetime)
FROM read_json_auto("s3://playground/raw/events.json", sample_size=204800)
```

file: stg_events.sql
```sql
{{
config(
format='parquet',
incremental_strategy='insert_overwrite',
incremental=True,
location='s3://playground/warehouse/stg_events',
materialized='external',
options={"partition_by": "dw_date", "overwrite_or_ignore": 1}
)
}}

SELECT * FROM {{ ref('raw_events') }}
```

### Log output/Screenshots

Runtime Error
Catalog Error: Table with name raw_events does not exist! Did you mean "pg_catalog.pg_views"? LINE 6: SELECT * FROM "memory"."main"."raw_events" ^

View Detailed Error 🚨
{
"code": -1,
"message": "Runtime Error\n Catalog Error: Table with name raw_events does not exist!\n Did you mean \"pg_catalog.pg_views\"?\n LINE 6: SELECT * FROM \"memory\".\"main\".\"raw_events\"\n ^",
"data": "\"Error: Runtime Error\\n Catalog Error: Table with name raw_events does not exist!\\n Did you mean \\\"pg_catalog.pg_views\\\"?\\n LINE 6: SELECT * FROM \\\"memory\\\".\\\"main\\\".\\\"raw_events\\\"\\n ^\\n\\tat DBTCoreProjectIntegration_1. (/Users/bumu/.vscode/extensions/innoverio.vscode-dbt-power-user-0.44.4/dist/extension.js:18686:127)\\n\\tat Generator.throw ()\\n\\tat rejected (/Users/bumu/.vscode/extensions/innoverio.vscode-dbt-power-user-0.44.4/dist/extension.js:27066:28)\""
}

### Operating System

macOS 14.6.1

### dbt version

1.8.5

### dbt Adapter

dbt-duckdb 1.8.3

### dbt Power User version

0.44.4

### Are you willing to submit PR?

- [ ] Yes I am willing to submit a PR!

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.