dbt-labs / dbt-labs/dbt-adapters

[Bug] dbt-athena is compiling non-specified sources

Open
#748 3 comments 0 reactions 0 assignees View on GitHub
pkg:dbt-athena
Dominant language
Python
Stars
233
Forks
362
Avg merge
3d 22h
Merged PRs (30d)
9

Description

### Is this a new bug in dbt-athena?

- [x] I believe this is a new bug in dbt-athena
- [x] I have searched the existing issues, and I could not find an existing issue for this bug

### Current Behavior

I have two major databases in my dbt project sources - Athena and Trino and two targets in projects profiles as shown below
```
version: 2

sources:
- name: trino
database: trino
schema: trino_prod
tables:
- name: xxxxxx
- name: athena
database: AwsDataCatalog
schema: athena
tables:
- name: xxxxx
-
```
```
trino:
target: prod
outputs:
prod:
type: trino
host: xxx.xxx.xxx.xxx
user: xxx
password: xxx
catalog: xxx
port: 8080
schema: xxx
threads: 1
method: ldap
http_scheme: https
athena:
target: prod
outputs:
prod:
type: athena
region_name: us-east-1
schema: xxx
database: awsdatacatalog
aws_profile_name: xxx-xxx-xxx
work_group: xxx-xx-xxx
s3_staging_dir: s3://xxx
s3_data_dir: s3://xxxx/xxx
s3_data_naming: table
s3_tmp_table_dir: s3://xxxxxx
poll_interval: 10
```

I can use ```dbt docs generate --select "source:trino" --profile trino``` to limit generating documentation for trino but I can't do the same for ```dbt docs generate --select "source:athena" --profile athena``` as this leads to generating documentation for source:trino ultimately leading to failure

```
13:26:16 Encountered an error while generating catalog: An error occurred (InvalidRequestException) when calling the GetDataCatalog operation: DataCatalog xxxxxx_xxxx was not found.
13:26:16 Encountered an error while generating catalog: An error occurred (InvalidRequestException) when calling the GetDataCatalog operation: DataCatalog xxxxxxxx_xxxx_xxxxxx was not found.
```

### Expected Behavior

When I run ```dbt docs generate --select "source:athena" --profile athena``` I would like to generate the documentation for objects that depend on "source:athena"

### Steps To Reproduce

1. Create a sources.yml that has Athena and Trino sources
2. Create a profiles.yml that has Athena and Trino targets
3. Run ```dbt docs generate --select "source:athena" --profile athena``` and it tries to generate the metadata for databases accessible via Trino as well.

### Relevant log output

```shell
13:26:16 Encountered an error while generating catalog: An error occurred (InvalidRequestException) when calling the GetDataCatalog operation: DataCatalog xxxxxx_xxxx was not found.
13:26:16 Encountered an error while generating catalog: An error occurred (InvalidRequestException) when calling the GetDataCatalog operation: DataCatalog xxxxxxxx_xxxx_xxxxxx was not found.
```

### Environment

```markdown
- OS: AL2
- Python: Python 3.11.11
- dbt-core: 1.7.15
- dbt-athena: 1.7.2
- dbt-trino: 1.7.1
```

### Additional Context

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.