dbt-labs / dbt-labs/dbt-adapters

[Bug] dbt-athena adapter generates malformed query when model uses bucket partition transform and batching

Open
#1,237 0 comments 0 reactions 0 assignees View on GitHub
triage:product type:bug
Dominant language
Python
Stars
233
Forks
362
Avg merge
3d 22h
Merged PRs (30d)
9

Description

### Is this a new bug?

- [x] I believe this is a new bug
- [x] I have searched the existing issues, and I could not find an existing issue for this bug

### Which packages are affected?

- [ ] dbt-adapters
- [ ] dbt-tests-adapter
- [x] dbt-athena
- [ ] dbt-athena-community
- [ ] dbt-bigquery
- [ ] dbt-postgres
- [ ] dbt-redshift
- [ ] dbt-snowflake
- [ ] dbt-spark

### Current Behavior

When doing an incremental build using an iceberg model that uses bucketing partition transformation, the query generated to insert a batch of records is malformed.

### Expected Behavior

The incremental build should succeed without throwing any error. I am not sure what form that query is supposed to take, though.

### Steps To Reproduce

Define an incremental model using iceberg with partitioning config like below:

```yml
partitioned_by:
- bucket(id,1000)
```
It succeeds on a full refresh, but on a subsequent incremental build I get this error:

```
16:45:57 Runtime Error in model *** (***)
An error occurred (InvalidRequestException) when calling the StartQueryExecution operation (reached max retries: 0): line 13:25: mismatched input 'and'. Expecting: ')', ,
```

When I view the logs I see this query being submitted - which fails since there is a missing field expression: `where ( and id IN...`
```sql
create table "awsdatacatalog"."***"."***__dbt_tmp"
with (
table_type='iceberg',
is_external=false,location='***',
partitioning=ARRAY['bucket(id, 1000)'],
write_compression='gzip',
format='parquet'
)
as
select "id", "***"
from "awsdatacatalog"."***"."***__dbt_tmp__tmp_not_partitioned"
where ( and id IN ('***', '***', '***')) or ( and id IN ('***', '***', '***', '***')) ...
```

### Relevant log output

```shell

```

### Environment

```markdown
- OS: MacOS 15.5
- Python: 3.11.9
- dbt-adapters: 1.16.0
- dbt-athena: 1.9.4
```

### Additional Context

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.