dbt-labs / dbt-labs/dbt-adapters

[Feature] Support Postgres `WITH` clause for index storage parameters

Open
#1,252 0 comments 0 reactions 0 assignees View on GitHub
triage:product type:enhancement
Dominant language
Python
Stars
233
Forks
362
Avg merge
3d 22h
Merged PRs (30d)
9

Description

### Is this your first time submitting a feature request?

- [x] I have read the [expectations for open source contributors](https://docs.getdbt.com/docs/contributing/oss-expectations)
- [x] I have searched the existing issues, and I could not find an existing issue for this feature
- [x] I am requesting a straightforward extension of existing dbt functionality, rather than a Big Idea better suited to a discussion

### Describe the feature

## Description

Currently, dbt-postgres supports basic index configuration including `columns`, `unique`, and `type`, but doesn't support PostgreSQL's `WITH` clause for specifying storage parameters. This limits the ability to optimise indexes for specific use cases.

This would really simplify using PostgreSQL's advanced indexing capabilities while maintaining the nicer declarative configuration approach (the example above is inline, but YAML is great).

## Current Behaviour

The current index configuration in dbt-postgres only supports (to my knowledge):

```jinja2
{{ config(indexes=[{'columns': ['column_a']}]) }}
```

This generates the following SQL (reformatted), which is handy as it operates on the `_tmp` table, not the final table (which post_hook does).

```sql
create index if not exists "dbe37318bd3277266e837eaa3633d372" on "example"."foo"."fct_foo__dbt_tmp" (column_a)
```

## Proposed Enhancement

Add support for a `with` parameter that allows passing custom storage parameters:

```yaml
indexes:
- columns: ['column_a']
type: 'gin'
with:
fastupdate: 'off'
- columns: ['title']
type: 'btree'
with:
fillfactor: 90
```

This would generate:
```sql
CREATE INDEX "locations_idx" ON "table_name" USING gin (locations) WITH (fastupdate = off, gin_pending_list_limit = 1024);
CREATE INDEX "title_idx" ON "table_name" USING btree (title) WITH (fillfactor = 70, deduplicate_items = off);
```

## Use Cases

1. **GIN Index Optimisation**: Disable `fastupdate` for better query performance on read-heavy tables
2. **B-tree Tuning**: Adjust `fillfactor` for tables with frequent inserts/updates
3. **Memory Management**: Set custom `gin_pending_list_limit` for large GIN indexes
4. **BRIN Configuration**: Set `pages_per_range` and `autosummarize` for time-series data
5. **GiST Optimisation**: Configure `buffering` behaviour for spatial indexes

## Ideas for with

The `with` parameter could accept:
- A dictionary/mapping of parameter names to values
- Automatic handling of parameter value types (boolean, integer, string)
- Validation against known PostgreSQL storage parameters (optional)

## Benefits

- Enables fine-tuning of PostgreSQL indexes for performance optimisation
- Maintains backward compatibility with existing configurations
- Provides access to PostgreSQL-specific features while staying within dbt's configuration paradigm
- Particularly valuable for data warehouses and analytics workloads where index performance is critical

## References

- [PostgreSQL CREATE INDEX documentation](https://www.postgresql.org/docs/16/sql-createindex.html#SQL-CREATEINDEX-STORAGE-PARAMETERS)
- [dbt-postgres index configuration](https://docs.getdbt.com/reference/resource-configs/postgres-configs#indexes)

### Describe alternatives you've considered

Post hooks, but this runs after the rename? Is it possible to do this BEFORE the rename of the table?

### Who will this benefit?

_No response_

### Are you interested in contributing this feature?

sure

### Anything else?

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.