dbt-labs / dbt-labs/dbt-adapters
[Feature] Support Postgres `WITH` clause for index storage parameters
- Dominant language
- Python
- Stars
- 233
- Forks
- 362
- Avg merge
- 3d 22h
- Merged PRs (30d)
- 9
Description
### Is this your first time submitting a feature request?
- [x] I have read the [expectations for open source contributors](https://docs.getdbt.com/docs/contributing/oss-expectations)
- [x] I have searched the existing issues, and I could not find an existing issue for this feature
- [x] I am requesting a straightforward extension of existing dbt functionality, rather than a Big Idea better suited to a discussion
### Describe the feature
## Description
Currently, dbt-postgres supports basic index configuration including `columns`, `unique`, and `type`, but doesn't support PostgreSQL's `WITH` clause for specifying storage parameters. This limits the ability to optimise indexes for specific use cases.
This would really simplify using PostgreSQL's advanced indexing capabilities while maintaining the nicer declarative configuration approach (the example above is inline, but YAML is great).
## Current Behaviour
The current index configuration in dbt-postgres only supports (to my knowledge):
```jinja2
{{ config(indexes=[{'columns': ['column_a']}]) }}
```
This generates the following SQL (reformatted), which is handy as it operates on the `_tmp` table, not the final table (which post_hook does).
```sql
create index if not exists "dbe37318bd3277266e837eaa3633d372" on "example"."foo"."fct_foo__dbt_tmp" (column_a)
```
## Proposed Enhancement
Add support for a `with` parameter that allows passing custom storage parameters:
```yaml
indexes:
- columns: ['column_a']
type: 'gin'
with:
fastupdate: 'off'
- columns: ['title']
type: 'btree'
with:
fillfactor: 90
```
This would generate:
```sql
CREATE INDEX "locations_idx" ON "table_name" USING gin (locations) WITH (fastupdate = off, gin_pending_list_limit = 1024);
CREATE INDEX "title_idx" ON "table_name" USING btree (title) WITH (fillfactor = 70, deduplicate_items = off);
```
## Use Cases
1. **GIN Index Optimisation**: Disable `fastupdate` for better query performance on read-heavy tables
2. **B-tree Tuning**: Adjust `fillfactor` for tables with frequent inserts/updates
3. **Memory Management**: Set custom `gin_pending_list_limit` for large GIN indexes
4. **BRIN Configuration**: Set `pages_per_range` and `autosummarize` for time-series data
5. **GiST Optimisation**: Configure `buffering` behaviour for spatial indexes
## Ideas for with
The `with` parameter could accept:
- A dictionary/mapping of parameter names to values
- Automatic handling of parameter value types (boolean, integer, string)
- Validation against known PostgreSQL storage parameters (optional)
## Benefits
- Enables fine-tuning of PostgreSQL indexes for performance optimisation
- Maintains backward compatibility with existing configurations
- Provides access to PostgreSQL-specific features while staying within dbt's configuration paradigm
- Particularly valuable for data warehouses and analytics workloads where index performance is critical
## References
- [PostgreSQL CREATE INDEX documentation](https://www.postgresql.org/docs/16/sql-createindex.html#SQL-CREATEINDEX-STORAGE-PARAMETERS)
- [dbt-postgres index configuration](https://docs.getdbt.com/reference/resource-configs/postgres-configs#indexes)
### Describe alternatives you've considered
Post hooks, but this runs after the rename? Is it possible to do this BEFORE the rename of the table?
### Who will this benefit?
_No response_
### Are you interested in contributing this feature?
sure
### Anything else?
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.