dbt-labs / dbt-labs/dbt-adapters
[Feature] Use atomic CREATE OR REPLACE VIEW for PostgreSQL view materialization
- Dominant language
- Python
- Stars
- 233
- Forks
- 362
- Avg merge
- 3d 22h
- Merged PRs (30d)
- 9
Description
## Problem Statement
The current view materialization uses a **rename-swap pattern** that creates a brief window where the view doesn't exist, causing downstream BI tools (like Sigma) to fail with errors like `"relation does not exist"`.
### Current Behavior
```sql
-- 1. Create intermediate view
CREATE VIEW my_view__dbt_tmp AS SELECT ...
-- 2. Rename existing → backup (⚠️ my_view GONE!)
ALTER VIEW my_view RENAME TO my_view__dbt_backup
-- 3. Rename intermediate → target
ALTER VIEW my_view__dbt_tmp RENAME TO my_view
-- 4. COMMIT
-- 5. DROP my_view__dbt_backup
```
Between steps 2 and 3, `my_view` doesn't exist. Concurrent queries fail.
### Impact
- BI tools (Sigma, Looker, Tableau) querying views during dbt runs get `"relation does not exist"` errors
- Users see intermittent dashboard failures
- This affects production systems with frequent dbt refreshes (e.g., hourly jobs)
## Proposed Solution
PostgreSQL supports atomic `CREATE OR REPLACE VIEW`. The dbt codebase **already has this macro**:
```sql
-- dbt/include/postgres/macros/relations/view/replace.sql
create or replace view {{ relation }}
as (
{{ sql }}
);
```
And PostgreSQL relations are marked as replaceable:
```python
# dbt/adapters/postgres/relation.py
replaceable_relations = frozenset({
RelationType.View, # ✅ Already marked as replaceable!
RelationType.Table,
})
```
**But the view materialization doesn't use it** - it uses the rename pattern instead.
### Requested Change
Add a configuration option to enable atomic view replacement for adapters that support it:
```yaml
# Option 1: Model-level config
{{ config(
materialized='view',
use_atomic_replace=true # New option
) }}
# Option 2: Project-level config in dbt_project.yml
models:
my_project:
+use_atomic_replace: true
```
When enabled on PostgreSQL (and other adapters that support it), use:
```sql
CREATE OR REPLACE VIEW my_view AS SELECT ...
```
This is a single atomic operation with **zero downtime**.
## Why Not Just Use CREATE OR REPLACE by Default?
I understand the rename pattern was chosen for cross-adapter consistency and safety. Some considerations:
1. **Column changes**: `CREATE OR REPLACE VIEW` in PostgreSQL fails if column count/types change. However, this is actually a feature - it prevents breaking downstream consumers.
2. **Adapter support**: Not all adapters support atomic replacement. A config option allows opting in where supported.
3. **Backward compatibility**: Making this opt-in preserves existing behavior.
## Workarounds Considered
- **Blue-green deployment**: Significant operational overhead
- **Custom materialization**: Requires maintaining custom code
- **Retry logic in BI tools**: Doesn't solve the root cause
- **Reduced job frequency**: Impacts data freshness
## Environment
- dbt-core version: 1.10.13
- dbt-postgres version: 1.10.x
- Database: PostgreSQL (Aurora)
## Related Code
The `get_replace_sql` macro in `macros/relations/replace.sql` already has logic to use atomic replacement when `can_be_replaced` is true - it just needs to be wired into the view materialization.
---
Thank you for considering this enhancement! Happy to provide more details or help with implementation.
Contributor guide
Assessment
This issue has not been assessed yet.