dbt-labs / dbt-labs/docs.getdbt.com
For dbt seeds and `csv` format of unit testing, empty strings (`""`) and `"null"` both convert to `NULL`
- Dominant language
- JavaScript
- Stars
- 215
- Forks
- 1.2k
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 130
Description
### Contributions
- [X] I have read the contribution docs, and understand what's expected of me.
### Link to the page on docs.getdbt.com requiring updates
https://docs.getdbt.com/docs/build/seeds
### What part(s) of the page would you like to see updated?
Many people are surprised about how CSV seeds treat certain values.
Namely, all of these will produce `NULL` values within the database when loaded via `dbt seed`:
`seeds/null_examples.csv`
```csv
type,value
doublequotes,""
empty,
spaces,
nothing
string_null,null
```
But these will produce not null strings (`` and `None` respectively):
`seeds/not_null_examples.csv`
```csv
type,value
single_quotes,''
string_none,None
```
### Example commands & output
```shell
dbt show -s not_null_examples --output json
```
```
16:57:53 Running with dbt=1.6.5
16:57:53 Registered adapter: spark=1.6.2
16:57:53 Found 3 seeds, 1 model, 0 sources, 0 exposures, 0 metrics, 398 macros, 0 groups, 0 semantic models
16:57:53
16:57:56 Concurrency: 1 threads (target='http')
16:57:56
16:57:56 1 of 1 START seed file dbt_dbeatty.not_null_examples ........................... [RUN]
16:58:03 1 of 1 OK loaded seed file dbt_dbeatty.not_null_examples ....................... [INSERT 2 in 6.89s]
16:58:03 {
"node": "not_null_examples",
"show": [
{
"type": "single_quotes",
"value": "''"
},
{
"type": "string_none",
"value": "None"
}
]
}
```
```shell
dbt show -s null_examples --output json
```
```
16:58:14 Running with dbt=1.6.5
16:58:15 Registered adapter: spark=1.6.2
16:58:15 Found 3 seeds, 1 model, 0 sources, 0 exposures, 0 metrics, 398 macros, 0 groups, 0 semantic models
16:58:15
16:58:17 Concurrency: 1 threads (target='http')
16:58:17
16:58:17 1 of 1 START seed file dbt_dbeatty.null_examples ............................... [RUN]
16:58:24 1 of 1 OK loaded seed file dbt_dbeatty.null_examples ........................... [INSERT 5 in 6.82s]
16:58:24 {
"node": "null_examples",
"show": [
{
"type": "doublequotes",
"value": null
},
{
"type": "empty",
"value": null
},
{
"type": "spaces",
"value": null
},
{
"type": "nothing",
"value": null
},
{
"type": "string_null",
"value": null
}
]
}
```
### Net consequence
There are two values that you can't load directly with CSV seeds:
1. An empty / blank string (`""` / `''`)
1. A string with the value `"null"`
In order to create those values, then we'd described a workaround at a high-level [here](https://github.com/dbt-labs/dbt-core/issues/4725#issuecomment-1047266185).
> you can opt for a "placeholder" empty string value (e.g. EMPTY), coupled with a staging model that wraps around the seed
👉 It might be handy for us to actually provide an example of that:
`seeds/sentinel_examples.csv`
```csv
type,value
sentinel_empty_string,EMPTY_STRING
sentinal_null_string,NULL_STRING
doublequotes,""
empty,
spaces,
nothing
string_null,null
single_quotes,''
string_none,None
```
`models/example_seed_wrapping.sql`
```sql
select
value as orginal_value,
case
when value = 'EMPTY_STRING' then ''
when value = 'NULL_STRING' then 'null'
else value
end as converted_value
from {{ ref('sentinel_examples') }}
```
```shell
dbt build -s +wrap_seed --full-refresh
dbt show -s wrap_seed --limit 10 --output json
```
Output
```json
{
"node": "wrap_seed",
"show": [
{
"type": "sentinel_empty_string",
"orginal_value": "EMPTY_STRING",
"converted_value": ""
},
{
"type": "sentinal_null_string",
"orginal_value": "NULL_STRING",
"converted_value": "null"
},
{
"type": "doublequotes",
"orginal_value": null,
"converted_value": null
},
{
"type": "empty",
"orginal_value": null,
"converted_value": null
},
{
"type": "spaces",
"orginal_value": null,
"converted_value": null
},
{
"type": "nothing",
"orginal_value": null,
"converted_value": null
},
{
"type": "string_null",
"orginal_value": null,
"converted_value": null
},
{
"type": "single_quotes",
"orginal_value": "''",
"converted_value": "''"
},
{
"type": "string_none",
"orginal_value": "None",
"converted_value": "None"
}
]
}
```
### Additional information
Here are some of the related issues and discussions:
- https://github.com/dbt-labs/dbt-core/issues/1466
- https://github.com/dbt-labs/dbt-core/issues/4725
- https://github.com/dbt-labs/dbt-adapters/issues/487
- https://github.com/dbt-labs/dbt-core/issues/9881
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.