apache / apache/datafusion-sqlparser-rs

ClickHouse: `Display` implementation converts types to uppercase, causing `UNKNOWN_TYPE` errors

Open
#2,153 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
3.5k
Forks
772
Avg merge
4d 9h
Merged PRs (30d)
17

Description

## Problem

ClickHouse data types are case-sensitive and require PascalCase (e.g., `String`, `Int32`, `Nullable`). However, the `sqlparser-rs` library's `Display` implementation for `DataType` converts certain types to uppercase, causing `UNKNOWN_TYPE` errors when round-tripping SQL through ClickHouse.

## Example
```rust
use sqlparser::dialect::ClickHouseDialect;
use sqlparser::parser::Parser;

let sql = "CREATE TABLE t (col Nullable(String))";
let dialect = ClickHouseDialect {};
let ast = Parser::parse_sql(&dialect, sql).unwrap();

// Round-trip: parse and convert back to string
let regenerated = ast[0].to_string();
// Result: "CREATE TABLE t (col Nullable(STRING))"
// ^^^^^^ uppercase!
```

When this regenerated SQL is executed against ClickHouse, it fails with:

```
Code: 47. DB::Exception: Unknown type STRING. (UNKNOWN_TYPE)
```

## Affected Types

| Type | Current Output | ClickHouse Requires |
|------|----------------|---------------------|
| `DataType::Int8` | `INT8` | `Int8` |
| `DataType::Int64` | `INT64` | `Int64` |
| `DataType::Float64` | `FLOAT64` | `Float64` |
| `DataType::String` | `STRING` | `String` |
| `DataType::Bool` | `BOOL` | `Bool` |
| `DataType::Date` | `DATE` | `Date` |
| `DataType::Datetime` | `DATETIME` | `DateTime` |

**Types already correct (PascalCase):**
- `Int16`, `Int32`, `Int128`, `Int256`
- `UInt8`, `UInt16`, `UInt32`, `UInt64`, `UInt128`, `UInt256`
- `Float32`
- `Nullable`, `LowCardinality`, `Array`, `Map`, `Tuple`, `Nested`

## Root Cause

The `Display` trait implementation for `DataType` uses uppercase for type names (e.g., `write!(f, "STRING")`), which is standard for most SQL dialects but incorrect for ClickHouse.

The challenge is that `Display` doesn't have access to dialect context, so it can't conditionally format based on the active dialect.

## The Problem with `Display`

Most users serialize SQL by calling `Display` on top-level AST types:

```rust
let ast = Parser::parse_sql(&dialect, sql).unwrap();
let regenerated = ast[0].to_string(); // Uses Display on Statement
// or
let regenerated = format!("{}", query); // Uses Display on Query
```

The `Display` trait signature doesn't allow passing context:
```rust
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result
```

## Proposed Solution

To solve this, we need dialect-aware serialization throughout the AST hierarchy:

Add `to_sql(&dyn Dialect)` to all AST types
- Add `to_sql(&dyn Dialect) -> String` method to `Statement`, `Query`, `Expr`, `ColumnDef`, and other AST types
- Each type's implementation calls `to_sql()` on its children, propagating the dialect
- Keep existing `Display` implementations unchanged for backwards compatibility

```rust
// Example usage after fix:
let ast = Parser::parse_sql(&dialect, sql).unwrap();
let regenerated = ast[0].to_sql(&dialect); // Correct PascalCase for ClickHouse
```

### Implementation Scope

The affected types include (non-exhaustive):
- `Statement` (top-level)
- `Query`, `SetExpr`, `Select`
- `Expr` (especially `Cast`, `TryCast`, `SafeCast`)
- `ColumnDef`, `ColumnOption`
- `TableConstraint`
- `AlterTableOperation`
- `FunctionArg`, `FunctionArgExpr`

This is a significant change but provides the cleanest API and maintains backwards compatibility.

## Workarounds

Currently, users must post-process the SQL string output to fix casing. See [514-labs/moosestack#3152](https://github.com/514-labs/moosestack/pull/3152) for an example regex-based workaround.

## References

- [ClickHouse Data Types Documentation](https://clickhouse.com/docs/en/sql-reference/data-types)
- Related workaround: [514-labs/moosestack#3152](https://github.com/514-labs/moosestack/pull/3152)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by inspecting the DataType Display implementation and the AST types listed in the issue, including Statement, Query, Expr, ColumnDef, and related nodes. Trace how serialization reaches nested data types, then assess the proposed dialect-aware to_sql API across the hierarchy. Done means the ClickHouse example round-trips with PascalCase types while existing Display behavior remains backward compatible.

Written by the indexing model from the issue text.

Assessment

Tech stack
clickhouse, rust
Domain
databases
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.