apache / apache/datafusion

`CREATE TABLE AS` should not inherit nullability from query

Open
#22,484 2 comments 0 reactions 1 assignee Claimed by @neilconway View on GitHub
bug
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Describe the bug

### Describe the bug

`CREATE TABLE AS SELECT ...` converts non-nullable properties of the SELECT into not-nullable constraints on the columns of the newly created table. This is defensible, but I think it's surprising, for two reasons:

1. "nullable=false" in the query just says that the optimizer can prove that the _query_ doesn't contain any nullable values in that column. That is distinct from saying that the newly created table ought to semantically have a NOT NULL constraint. You can imagine plenty of scenarios where the initial values of a table are populated with `CREATE TABLE AS SELECT ...` that doesn't have any NULL values, but then subsequent DML on the table expects to be able to insert NULLs.
2. Whether an expression in a query is not-nullable depends on the vaguaries of query optimization, which change between releases. Adding this instability to the output of `CREATE TABLE AS SELECT ...` seems undesirable.

The current behavior is also incompatible with Postgres and DuckDB, although that shouldn't necessarily be determinative.

### To Reproduce

```
> CREATE OR REPLACE TABLE t(a int) AS SELECT 1 FROM VALUES (5);
0 row(s) fetched.
Elapsed 0.012 seconds.

> \d t
+---------------+--------------+------------+-------------+-----------+-------------+
| table_catalog | table_schema | table_name | column_name | data_type | is_nullable |
+---------------+--------------+------------+-------------+-----------+-------------+
| datafusion | public | t | a | Int32 | NO |
+---------------+--------------+------------+-------------+-----------+-------------+
1 row(s) fetched.
Elapsed 0.010 seconds.
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.