dbt-labs / dbt-labs/dbt

[CT-3124] [Bug] Compilation Error when altering an index on a materialized view

Open
#8,655 3 comments 2 reactions 0 assignees View on GitHub
area:adapters engine:v1 materialized_views type:bug
Dominant language
Rust
Stars
13.8k
Forks
2.6k
Avg merge
21h 31m
Merged PRs (30d)
56

Description

### Is this a new bug in dbt-core?

- [X] I believe this is a new bug in dbt-core
- [X] I have searched the existing issues, and I could not find an existing issue for this bug

### Current Behavior

When changing an [index](https://docs.getdbt.com/reference/resource-configs/postgres-configs#indexes) definition with dbt-postgres, it raises an error.

### Expected Behavior

I was expecting no errors when running dbt, and that the index on the materialized view would be altered.

### Steps To Reproduce

Using dbt-core~1.6 with postgres:

Start with this single model in `models/my_mv.sql`:
```sql
{{ config(
materialized="materialized_view",
indexes=[
{'columns': ['column_a'], 'type': 'hash'},
],
on_configuration_change="apply",
)}}

select 1 as column_a, 2 as column_b
```

Then this will work fine:
```shell
dbt run -s my_mv
```

But if the index is altered with a different column name within `models/my_mv.sql` like this:
```sql
{{ config(
materialized="materialized_view",
indexes=[
{'columns': ['column_b'], 'type': 'hash'},
],
on_configuration_change="apply",
)}}

select 1 as column_a, 2 as column_b
```

Then this will yield a Compilation Error:
```shell
dbt run -s my_mv
```

### Relevant log output

Works fine on the initial run:
```shell
(postgres_1.6) $ dbt run -s my_mv
22:39:19 Running with dbt=1.6.0
22:39:19 Registered adapter: postgres=1.6.0
22:39:19 Unable to do partial parsing because of a version mismatch
22:39:20 Found 2 models, 0 sources, 0 exposures, 0 metrics, 349 macros, 0 groups, 0 semantic models
22:39:20
22:39:20 Concurrency: 5 threads (target='postgres')
22:39:20
22:39:20 1 of 1 START sql materialized_view model dbt_dbeatty.my_mv ..................... [RUN]
22:39:20 1 of 1 OK created sql materialized_view model dbt_dbeatty.my_mv ................ [REFRESH MATERIALIZED VIEW in 0.12s]
22:39:20
22:39:20 Finished running 1 materialized_view model in 0 hours 0 minutes and 0.31 seconds (0.31s).
22:39:20
22:39:20 Completed successfully
22:39:20
22:39:20 Done. PASS=1 WARN=0 ERROR=0 SKIP=0 TOTAL=1
```

Breaks when re-running after the index definition is changed:

```shell
(postgres_1.6) $ dbt run -s my_mv
22:39:27 Running with dbt=1.6.0
22:39:27 Registered adapter: postgres=1.6.0
22:39:27 Found 2 models, 0 sources, 0 exposures, 0 metrics, 349 macros, 0 groups, 0 semantic models
22:39:27
22:39:27 Concurrency: 5 threads (target='postgres')
22:39:27
22:39:27 1 of 1 START sql materialized_view model dbt_dbeatty.my_mv ..................... [RUN]
22:39:27 1 of 1 ERROR creating sql materialized_view model dbt_dbeatty.my_mv ............ [ERROR in 0.09s]
22:39:28
22:39:28 Finished running 1 materialized_view model in 0 hours 0 minutes and 0.27 seconds (0.27s).
22:39:28
22:39:28 Completed with 1 error and 0 warnings:
22:39:28
22:39:28 Compilation Error in model my_mv (models/my_mv.sql)
22:39:28 Could not parse index config: at path []: Undefined is not of type 'object'
22:39:28
22:39:28 > in macro postgres__get_create_index_sql (macros/adapters.sql)
22:39:28 > called by macro postgres__update_indexes_on_materialized_view (macros/materializations/materialized_view.sql)
22:39:28 > called by macro postgres__get_alter_materialized_view_as_sql (macros/materializations/materialized_view.sql)
22:39:28 > called by macro get_alter_materialized_view_as_sql (macros/materializations/models/materialized_view/alter_materialized_view.sql)
22:39:28 > called by macro materialized_view_get_build_sql (macros/materializations/models/materialized_view/materialized_view.sql)
22:39:28 > called by macro materialization_materialized_view_default (macros/materializations/models/materialized_view/materialized_view.sql)
22:39:28 > called by model my_mv (models/my_mv.sql)
22:39:28
22:39:28 Done. PASS=0 WARN=0 ERROR=1 SKIP=0 TOTAL=1
```

### Environment

```markdown
- OS: macOS
- Python: 3.8.13
- dbt: 1.6.0
```

### Which database adapter are you using with dbt?

postgres

### Additional Context

The presumed problem/solution is described in [this comment](https://github.com/dbt-labs/dbt-core/issues/8655#issuecomment-1722335987).

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.