apache / apache/datafusion-sqlparser-rs
ClickHouse: `Display` implementation converts types to uppercase, causing `UNKNOWN_TYPE` errors
- 主要语言
- Rust
- 星标
- 3.5k
- 派生
- 772
- 平均合并
- 4 天 9 小时
- 30 天内合并 PR
- 17
描述
## Problem
ClickHouse data types are case-sensitive and require PascalCase (e.g., `String`, `Int32`, `Nullable`). However, the `sqlparser-rs` library's `Display` implementation for `DataType` converts certain types to uppercase, causing `UNKNOWN_TYPE` errors when round-tripping SQL through ClickHouse.
## Example
```rust
use sqlparser::dialect::ClickHouseDialect;
use sqlparser::parser::Parser;
let sql = "CREATE TABLE t (col Nullable(String))";
let dialect = ClickHouseDialect {};
let ast = Parser::parse_sql(&dialect, sql).unwrap();
// Round-trip: parse and convert back to string
let regenerated = ast[0].to_string();
// Result: "CREATE TABLE t (col Nullable(STRING))"
// ^^^^^^ uppercase!
```
When this regenerated SQL is executed against ClickHouse, it fails with:
```
Code: 47. DB::Exception: Unknown type STRING. (UNKNOWN_TYPE)
```
## Affected Types
| Type | Current Output | ClickHouse Requires |
|------|----------------|---------------------|
| `DataType::Int8` | `INT8` | `Int8` |
| `DataType::Int64` | `INT64` | `Int64` |
| `DataType::Float64` | `FLOAT64` | `Float64` |
| `DataType::String` | `STRING` | `String` |
| `DataType::Bool` | `BOOL` | `Bool` |
| `DataType::Date` | `DATE` | `Date` |
| `DataType::Datetime` | `DATETIME` | `DateTime` |
**Types already correct (PascalCase):**
- `Int16`, `Int32`, `Int128`, `Int256`
- `UInt8`, `UInt16`, `UInt32`, `UInt64`, `UInt128`, `UInt256`
- `Float32`
- `Nullable`, `LowCardinality`, `Array`, `Map`, `Tuple`, `Nested`
## Root Cause
The `Display` trait implementation for `DataType` uses uppercase for type names (e.g., `write!(f, "STRING")`), which is standard for most SQL dialects but incorrect for ClickHouse.
The challenge is that `Display` doesn't have access to dialect context, so it can't conditionally format based on the active dialect.
## The Problem with `Display`
Most users serialize SQL by calling `Display` on top-level AST types:
```rust
let ast = Parser::parse_sql(&dialect, sql).unwrap();
let regenerated = ast[0].to_string(); // Uses Display on Statement
// or
let regenerated = format!("{}", query); // Uses Display on Query
```
The `Display` trait signature doesn't allow passing context:
```rust
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result
```
## Proposed Solution
To solve this, we need dialect-aware serialization throughout the AST hierarchy:
Add `to_sql(&dyn Dialect)` to all AST types
- Add `to_sql(&dyn Dialect) -> String` method to `Statement`, `Query`, `Expr`, `ColumnDef`, and other AST types
- Each type's implementation calls `to_sql()` on its children, propagating the dialect
- Keep existing `Display` implementations unchanged for backwards compatibility
```rust
// Example usage after fix:
let ast = Parser::parse_sql(&dialect, sql).unwrap();
let regenerated = ast[0].to_sql(&dialect); // Correct PascalCase for ClickHouse
```
### Implementation Scope
The affected types include (non-exhaustive):
- `Statement` (top-level)
- `Query`, `SetExpr`, `Select`
- `Expr` (especially `Cast`, `TryCast`, `SafeCast`)
- `ColumnDef`, `ColumnOption`
- `TableConstraint`
- `AlterTableOperation`
- `FunctionArg`, `FunctionArgExpr`
This is a significant change but provides the cleanest API and maintains backwards compatibility.
## Workarounds
Currently, users must post-process the SQL string output to fix casing. See [514-labs/moosestack#3152](https://github.com/514-labs/moosestack/pull/3152) for an example regex-based workaround.
## References
- [ClickHouse Data Types Documentation](https://clickhouse.com/docs/en/sql-reference/data-types)
- Related workaround: [514-labs/moosestack#3152](https://github.com/514-labs/moosestack/pull/3152)
贡献指南
这个仓库没有索引到贡献指南
调研方向
首先检查 DataType Display 的实现,以及 issue 中列出的 AST 类型,包括 Statement、Query、Expr、ColumnDef 和相关节点。跟踪序列化如何处理嵌套数据类型,然后评估提议的、面向方言的 to_sql API 在整个层次结构中的适用性。当 ClickHouse 示例使用 PascalCase 类型完成往返,并且现有 Display 行为仍保持向后兼容时,即表示完成。
由索引模型根据 Issue 内容生成。
评估
- 技术栈
- clickhouse, rust
- 领域
- databases
- Issue 类型
- 缺陷
- 难度
- 5/5
- 预计耗时
- 一周以上
- 活跃度
- 停滞
- 描述清晰度
- 基本清楚
- 新手友好度
- 35/100