dbt-labs / dbt-labs/dbt-adapters
[Bug] dbt run against spark profile checks for database even in unrelated sources
- Dominant language
- Python
- Stars
- 233
- Forks
- 362
- Avg merge
- 3d 22h
- Merged PRs (30d)
- 9
Description
### Is this a new bug?
- [x] I believe this is a new bug
- [x] I have searched the existing issues, and I could not find an existing issue for this bug
### Which packages are affected?
- [ ] dbt-adapters
- [ ] dbt-tests-adapter
- [ ] dbt-athena
- [ ] dbt-athena-community
- [ ] dbt-bigquery
- [ ] dbt-postgres
- [ ] dbt-redshift
- [ ] dbt-snowflake
- [x] dbt-spark
### Current Behavior
sources.yml
```
- name: spark
description: glue default database
schema: default
tables:
- name: fact_customer_item_dly
description: fact_customer_item_dly
```
profiles.yml
```
sparkprofile:
target: dev
outputs:
dev:
type: spark
method: thrift
host: localhost
port: 10001
schema: dev
```
spark_test.sql
```
{{ config(
materialized='table',
file_format='parquet'
) }}
select count(*) from {{ source('spark' ,'fact_customer_item_dly') }}
```
`.venv/bin/dbt run --select spark_test.sql --profile sparkprofile` errors with Cannot set database in spark!
### Expected Behavior
With the above sources.yml, profiles.yml `.venv/bin/dbt run --select spark_test.sql --profile sparkprofile` should run without any error as source with name spark do not have any database specified
### Steps To Reproduce
Files sources.yml and profiles.yml should be as above
### Relevant log output
```shell
N/A
```
### Environment
```markdown
- OS: MACOS
- Python: 3.11.11
- dbt-spark: 1.8.0
```
### Additional Context
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.