dbt-labs / dbt-labs/dbt

Rationalize quoting configs + properties

Open
#2,986 18 comments 6 reactions 0 assignees View on GitHub
engine:v1 paper_cut type:feature
Dominant language
Rust
Stars
13.8k
Forks
2.6k
Avg merge
21h 31m
Merged PRs (30d)
56

Description

### Describe the feature

_Picks up from issues like #2468 and #2975, which are narrower in scope and offer more straightforward near-term fixes_

- Why do we call it [`quoting`](https://docs.getdbt.com/reference/project-configs/quoting/) when configuring database/schema/identifier names, but then [`quote`](https://docs.getdbt.com/reference/resource-properties/quote/) when describing properties of column names?
- Why does each adapter have to implement its quoting character sort-of twice? (#2243)
- There's also [`quote_columns`](https://docs.getdbt.com/reference/resource-configs/quote_columns/), which is a seed-only config item that lives on its own level but surely belongs inside the `quoting` config item (or will it be `quote`?!) as `columns`
- Not to mention [`quoted`](https://docs.getdbt.com/reference/dbt-classes/#properties), which while not itself a config, returns the quoted version of a column name from a Relation based on the configs above

Instead, we should have a single config/property, and I think it should be `quote`. This would take over from the current project-level `quoting` config:
```
quote:
database: true|false # or `project` on dbt-bigquery
schema: true|false # or `dataset` on dbt-bigquery
identifier: true|false
columns: true|false
```

The `quote: {columns: true}` would also replace `quote_columns` as a bespoke config for seeds. If that config is specified in `dbt_project.yml`, it can be superseded by:
- setting `quote: {}` inside the `config()` block for a specific moel
- `quote: true|false` set for a specific column in `models/*.yml` (it's implied that this really means `quote: {column: true}`
- in a post-#2401 world, a `model` can set its `quote: {}` config within `models/*.yml`, too

If `quote` is not set, it falls back to the default behavior of the adapter plugin, which also sets the character used for quoting (almost always `"` or ``` ` ```).

### Questions

Here's what we have in the [docs FAQs](https://docs.getdbt.com/docs/building-a-dbt-project/using-sources/) today for sources:

> By default, dbt will not quote the database, schema, or identifier for the source tables that you've specified.

Should sources start respecting project-level `quote` settings? Or they continue to act independently, but we should enable turning this config-property on or off for _all_ sources in `dbt_project.yml`:
```yml
sources:
quote:
schema: true
```

### Describe alternatives you've considered
Retaining all of these configs/properties/adapter methods and documenting them exceptionally well so as to avoid confusion

### Additional context
This isn't specific to any one database, though it is likely most helpful on databases that support special characters if quoted (Postgres, Redshift) or are particularly sensitive to quoting (Snowflake).

There's a round-up of all the known documentation related to quoting in https://github.com/dbt-labs/docs.getdbt.com/issues/3518.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.