dbt-labs / dbt-labs/dbt-adapters

[Bug] on_schema_change failure with alternate schema

Open
#411 3 comments 0 reactions 0 assignees View on GitHub
pkg:dbt-athena triage:product type:bug
Dominant language
Python
Stars
233
Forks
362
Avg merge
3d 22h
Merged PRs (30d)
9

Description

### Is this a new bug in dbt-athena?

- [X] I believe this is a new bug in dbt-athena
- [X] I have searched the existing issues, and I could not find an existing issue for this bug

### Current Behavior

I have an incremental table `int_nightly_sales`, that is an intermediate table in our workflow.

When attempting to add a column to this table, I get a failure:

```
16:17:13 dbt.adapters.athena.constants adapter: Error running SQL: alter table `base_db_intermediate`.`int_nightly_sales`
replace columns ()

16:17:13 On model.daily_sales_sync.int_nightly_sales: Close
16:17:13 Runtime Error in model int_nightly_sales (models/intermediate/upload/int_nightly_sales.sql)
FAILED: SemanticException [Error 10072]: Database does not exist: base_db

```

The table should actually exist on: `base_db_intermediate` as called out in the config:

```
{{ config(
materialized='incremental',
incremental_strategy='append',
partitioned_by=['processed_on'],
on_schema_change='sync_all_columns',
schema='intermediate'
) }}
```

I dug through the adapter source and could not find a culprit on this, but the odd thing is the first message references the correct db, where the semantic error thrown references the base_db (which does not actually contain anything at all).

### Expected Behavior

The table exists on the proper database in athena, the adapter should reference the correct database instead of the base database for `on_schema_change`.

### Steps To Reproduce

1. Create an athena dbt project with a base `schema` (`base`)
2. Create an incremental model with a different `schema` (`sub`) -> `base_sub`
3. `dbt build`
4. Add a column to the model
5. `dbt build`

This should fail as described.

### Environment

```markdown
- OS: Mint
- Python: 3.10.12
- dbt: 1.8.4
- dbt-athena-community: 1.8.4
```

### Additional Context

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.