apache / apache/datafusion

Pass CreateView options from sqlparser to logical_plan

Open
#12,228 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Is your feature request related to a problem or challenge?

I'm trying to implement Iceberg (Materialized) Views with Datafusion. For the 'CREATE VIEW' statement I require additional information about the storage location etc. Currently the LogicalPlan node for CreateView looks like this:

```rust
/// Creates a view.
#[derive(Clone, PartialEq, Eq, Hash)]
pub struct CreateView {
/// The table name
pub name: TableReference,
/// The logical plan
pub input: Arc,
/// Option to not error if table already exists
pub or_replace: bool,
/// SQL used to create the view, if available
pub definition: Option,
}
```

It doesn't contain any options passed to the view. This makes sense because Datafusion doesn't support any options. The ast node from the sqlparser crate looks like this:

```rust
/// ```sql
/// CREATE VIEW
/// ```
CreateView {
or_replace: bool,
materialized: bool,
/// View name
name: ObjectName,
columns: Vec,
query: Box,
options: CreateTableOptions,
cluster_by: Vec,
/// Snowflake: Views can have comments in Snowflake.
///
comment: Option,
/// if true, has RedShift [`WITH NO SCHEMA BINDING`] clause
with_no_schema_binding: bool,
/// if true, has SQLite `IF NOT EXISTS` clause
if_not_exists: bool,
/// if true, has SQLite `TEMP` or `TEMPORARY` clause
temporary: bool,
}
```

It contains an "options" field where additional options can be defined which I could use to implement the Iceberg Views.

### Describe the solution you'd like

It would be great if we could introduce a new "options" field for the CreateView LogicalPlan node, like so:

```rust
/// Creates a view.
#[derive(Clone, PartialEq, Eq, Hash)]
pub struct CreateView {
/// The table name
pub name: TableReference,
/// The logical plan
pub input: Arc,
/// Option to not error if table already exists
pub or_replace: bool,
/// SQL used to create the view, if available
pub definition: Option,
/// CreateView options
pub options: Option>
}
```

This way we could pass the SQL options from the ast to the LogicalPlan node.

### Describe alternatives you've considered

The alternative would be to define a new UserDefinedLogicalNode and implement a new SQL planner around SqlToRel. However, these could not be used with the Datafusion SessionContext and/or SessionState limiting the ability to integrate it with Datafusion.

### Additional context

_No response_

Contributor guide

Open the contributing guide

Research direction

Start at the CreateView LogicalPlan node and the SqlToRel path that converts the sqlparser CREATE VIEW AST, then trace how plans move through SessionContext and SessionState. Preserve the AST's view options in the logical plan and verify that CREATE VIEW planning through those entry points retains them.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust, sql
Domain
databases
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.