dbt-labs / dbt-labs/dbt-adapters
[Bug] dbt-athena adapter generates malformed query when model uses bucket partition transform and batching
- Dominant language
- Python
- Stars
- 233
- Forks
- 362
- Avg merge
- 3d 22h
- Merged PRs (30d)
- 9
Description
### Is this a new bug?
- [x] I believe this is a new bug
- [x] I have searched the existing issues, and I could not find an existing issue for this bug
### Which packages are affected?
- [ ] dbt-adapters
- [ ] dbt-tests-adapter
- [x] dbt-athena
- [ ] dbt-athena-community
- [ ] dbt-bigquery
- [ ] dbt-postgres
- [ ] dbt-redshift
- [ ] dbt-snowflake
- [ ] dbt-spark
### Current Behavior
When doing an incremental build using an iceberg model that uses bucketing partition transformation, the query generated to insert a batch of records is malformed.
### Expected Behavior
The incremental build should succeed without throwing any error. I am not sure what form that query is supposed to take, though.
### Steps To Reproduce
Define an incremental model using iceberg with partitioning config like below:
```yml
partitioned_by:
- bucket(id,1000)
```
It succeeds on a full refresh, but on a subsequent incremental build I get this error:
```
16:45:57 Runtime Error in model *** (***)
An error occurred (InvalidRequestException) when calling the StartQueryExecution operation (reached max retries: 0): line 13:25: mismatched input 'and'. Expecting: ')', ,
```
When I view the logs I see this query being submitted - which fails since there is a missing field expression: `where ( and id IN...`
```sql
create table "awsdatacatalog"."***"."***__dbt_tmp"
with (
table_type='iceberg',
is_external=false,location='***',
partitioning=ARRAY['bucket(id, 1000)'],
write_compression='gzip',
format='parquet'
)
as
select "id", "***"
from "awsdatacatalog"."***"."***__dbt_tmp__tmp_not_partitioned"
where ( and id IN ('***', '***', '***')) or ( and id IN ('***', '***', '***', '***')) ...
```
### Relevant log output
```shell
```
### Environment
```markdown
- OS: MacOS 15.5
- Python: 3.11.9
- dbt-adapters: 1.16.0
- dbt-athena: 1.9.4
```
### Additional Context
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.