Pass CreateView options from sqlparser to logical_plan
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
### Is your feature request related to a problem or challenge?
I'm trying to implement Iceberg (Materialized) Views with Datafusion. For the 'CREATE VIEW' statement I require additional information about the storage location etc. Currently the LogicalPlan node for CreateView looks like this:
```rust
/// Creates a view.
#[derive(Clone, PartialEq, Eq, Hash)]
pub struct CreateView {
/// The table name
pub name: TableReference,
/// The logical plan
pub input: Arc,
/// Option to not error if table already exists
pub or_replace: bool,
/// SQL used to create the view, if available
pub definition: Option,
}
```
It doesn't contain any options passed to the view. This makes sense because Datafusion doesn't support any options. The ast node from the sqlparser crate looks like this:
```rust
/// ```sql
/// CREATE VIEW
/// ```
CreateView {
or_replace: bool,
materialized: bool,
/// View name
name: ObjectName,
columns: Vec,
query: Box,
options: CreateTableOptions,
cluster_by: Vec,
/// Snowflake: Views can have comments in Snowflake.
///
comment: Option,
/// if true, has RedShift [`WITH NO SCHEMA BINDING`] clause
with_no_schema_binding: bool,
/// if true, has SQLite `IF NOT EXISTS` clause
if_not_exists: bool,
/// if true, has SQLite `TEMP` or `TEMPORARY` clause
temporary: bool,
}
```
It contains an "options" field where additional options can be defined which I could use to implement the Iceberg Views.
### Describe the solution you'd like
It would be great if we could introduce a new "options" field for the CreateView LogicalPlan node, like so:
```rust
/// Creates a view.
#[derive(Clone, PartialEq, Eq, Hash)]
pub struct CreateView {
/// The table name
pub name: TableReference,
/// The logical plan
pub input: Arc,
/// Option to not error if table already exists
pub or_replace: bool,
/// SQL used to create the view, if available
pub definition: Option,
/// CreateView options
pub options: Option>
}
```
This way we could pass the SQL options from the ast to the LogicalPlan node.
### Describe alternatives you've considered
The alternative would be to define a new UserDefinedLogicalNode and implement a new SQL planner around SqlToRel. However, these could not be used with the Datafusion SessionContext and/or SessionState limiting the ability to integrate it with Datafusion.
### Additional context
_No response_
Contributor guide
Research direction
Start at the CreateView LogicalPlan node and the SqlToRel path that converts the sqlparser CREATE VIEW AST, then trace how plans move through SessionContext and SessionState. Preserve the AST's view options in the logical plan and verify that CREATE VIEW planning through those entry points retains them.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust, sql
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 32/100