dbt-labs / dbt-labs/dbt-adapters

[Feature] Use atomic CREATE OR REPLACE VIEW for PostgreSQL view materialization

Open
#1,680 2 comments 1 reaction 0 assignees View on GitHub
triage:product
Dominant language
Python
Stars
233
Forks
362
Avg merge
3d 22h
Merged PRs (30d)
9

Description

## Problem Statement

The current view materialization uses a **rename-swap pattern** that creates a brief window where the view doesn't exist, causing downstream BI tools (like Sigma) to fail with errors like `"relation does not exist"`.

### Current Behavior

```sql
-- 1. Create intermediate view
CREATE VIEW my_view__dbt_tmp AS SELECT ...

-- 2. Rename existing → backup (⚠️ my_view GONE!)
ALTER VIEW my_view RENAME TO my_view__dbt_backup

-- 3. Rename intermediate → target
ALTER VIEW my_view__dbt_tmp RENAME TO my_view

-- 4. COMMIT
-- 5. DROP my_view__dbt_backup
```

Between steps 2 and 3, `my_view` doesn't exist. Concurrent queries fail.

### Impact

- BI tools (Sigma, Looker, Tableau) querying views during dbt runs get `"relation does not exist"` errors
- Users see intermittent dashboard failures
- This affects production systems with frequent dbt refreshes (e.g., hourly jobs)

## Proposed Solution

PostgreSQL supports atomic `CREATE OR REPLACE VIEW`. The dbt codebase **already has this macro**:

```sql
-- dbt/include/postgres/macros/relations/view/replace.sql
create or replace view {{ relation }}
as (
{{ sql }}
);
```

And PostgreSQL relations are marked as replaceable:

```python
# dbt/adapters/postgres/relation.py
replaceable_relations = frozenset({
RelationType.View, # ✅ Already marked as replaceable!
RelationType.Table,
})
```

**But the view materialization doesn't use it** - it uses the rename pattern instead.

### Requested Change

Add a configuration option to enable atomic view replacement for adapters that support it:

```yaml
# Option 1: Model-level config
{{ config(
materialized='view',
use_atomic_replace=true # New option
) }}

# Option 2: Project-level config in dbt_project.yml
models:
my_project:
+use_atomic_replace: true
```

When enabled on PostgreSQL (and other adapters that support it), use:
```sql
CREATE OR REPLACE VIEW my_view AS SELECT ...
```

This is a single atomic operation with **zero downtime**.

## Why Not Just Use CREATE OR REPLACE by Default?

I understand the rename pattern was chosen for cross-adapter consistency and safety. Some considerations:

1. **Column changes**: `CREATE OR REPLACE VIEW` in PostgreSQL fails if column count/types change. However, this is actually a feature - it prevents breaking downstream consumers.

2. **Adapter support**: Not all adapters support atomic replacement. A config option allows opting in where supported.

3. **Backward compatibility**: Making this opt-in preserves existing behavior.

## Workarounds Considered

- **Blue-green deployment**: Significant operational overhead
- **Custom materialization**: Requires maintaining custom code
- **Retry logic in BI tools**: Doesn't solve the root cause
- **Reduced job frequency**: Impacts data freshness

## Environment

- dbt-core version: 1.10.13
- dbt-postgres version: 1.10.x
- Database: PostgreSQL (Aurora)

## Related Code

The `get_replace_sql` macro in `macros/relations/replace.sql` already has logic to use atomic replacement when `can_be_replaced` is true - it just needs to be wired into the view materialization.

---

Thank you for considering this enhancement! Happy to provide more details or help with implementation.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.