apache / apache/datafusion

Ability to configure custom SQL parser dialects on session config

Open
#23,109 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Is your feature request related to a problem or challenge?

With https://github.com/apache/datafusion-sqlparser-rs/pull/2174, it is now possible to safely derive custom dialects from existing ones, without worrying about breaking hardcoded `dialect_of!` calls (https://github.com/apache/datafusion-sqlparser-rs/issues/1430):

```python
derive_dialect!(MyCustomDialect, PostgreSqlDialect,
overrides = {
supports_from_first_select = true,
},
preserve_type_id = true,
);
```

Datafusion allows configuring the dialect in the [`SessionConfig`](https://docs.rs/datafusion/latest/datafusion/execution/context/struct.SessionConfig.html), however only a [fixed list of values](https://docs.rs/datafusion/latest/datafusion/common/config/enum.Dialect.html) is allowed.

Would it be possible to allow setting our own dialect instance in the config? Perhaps with a `with_dialect()` method on the `SessionConfig`?

There might be limitations I'm missing (e.g. the underlying [`SqlParserOptions`](https://docs.rs/datafusion/latest/datafusion/config/struct.SqlParserOptions.html) might need to be serializable with atomic values, etc).

### Describe the solution you'd like

_No response_

### Describe alternatives you've considered

Extending the `RelationPlanner`: https://datafusion.apache.org/blog/2026/01/12/extending-sql/ (doesn't seem applicable).

[Defining a custom SQL parser](https://github.com/apache/datafusion/blob/main/datafusion-examples/examples/sql_ops/custom_sql_parser.rs), but not clear how to configure DF to use it.

### Additional context

_No response_

Contributor guide

Open the contributing guide

Research direction

Read SessionConfig, Dialect, and SqlParserOptions to understand how the fixed dialect values reach SQL parsing. Compare this with the custom_sql_parser.rs example and determine whether a custom dialect can be carried through session configuration. Done should mean a custom dialect can be configured and used by DataFusion's SQL parser without relying on the fixed Dialect list.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust, sql
Domain
databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.