apache / apache/datafusion-sqlparser-rs

ClickHouse: `Display` implementation converts types to uppercase, causing `UNKNOWN_TYPE` errors

Abierto
#2,153 0 comentarios 0 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
Rust
Estrellas
3.5k
Forks
772
Merge medio
4 d 9 h
PR fusionados (30 d)
17

Descripción

## Problem

ClickHouse data types are case-sensitive and require PascalCase (e.g., `String`, `Int32`, `Nullable`). However, the `sqlparser-rs` library's `Display` implementation for `DataType` converts certain types to uppercase, causing `UNKNOWN_TYPE` errors when round-tripping SQL through ClickHouse.

## Example
```rust
use sqlparser::dialect::ClickHouseDialect;
use sqlparser::parser::Parser;

let sql = "CREATE TABLE t (col Nullable(String))";
let dialect = ClickHouseDialect {};
let ast = Parser::parse_sql(&dialect, sql).unwrap();

// Round-trip: parse and convert back to string
let regenerated = ast[0].to_string();
// Result: "CREATE TABLE t (col Nullable(STRING))"
// ^^^^^^ uppercase!
```

When this regenerated SQL is executed against ClickHouse, it fails with:

```
Code: 47. DB::Exception: Unknown type STRING. (UNKNOWN_TYPE)
```

## Affected Types

| Type | Current Output | ClickHouse Requires |
|------|----------------|---------------------|
| `DataType::Int8` | `INT8` | `Int8` |
| `DataType::Int64` | `INT64` | `Int64` |
| `DataType::Float64` | `FLOAT64` | `Float64` |
| `DataType::String` | `STRING` | `String` |
| `DataType::Bool` | `BOOL` | `Bool` |
| `DataType::Date` | `DATE` | `Date` |
| `DataType::Datetime` | `DATETIME` | `DateTime` |

**Types already correct (PascalCase):**
- `Int16`, `Int32`, `Int128`, `Int256`
- `UInt8`, `UInt16`, `UInt32`, `UInt64`, `UInt128`, `UInt256`
- `Float32`
- `Nullable`, `LowCardinality`, `Array`, `Map`, `Tuple`, `Nested`

## Root Cause

The `Display` trait implementation for `DataType` uses uppercase for type names (e.g., `write!(f, "STRING")`), which is standard for most SQL dialects but incorrect for ClickHouse.

The challenge is that `Display` doesn't have access to dialect context, so it can't conditionally format based on the active dialect.

## The Problem with `Display`

Most users serialize SQL by calling `Display` on top-level AST types:

```rust
let ast = Parser::parse_sql(&dialect, sql).unwrap();
let regenerated = ast[0].to_string(); // Uses Display on Statement
// or
let regenerated = format!("{}", query); // Uses Display on Query
```

The `Display` trait signature doesn't allow passing context:
```rust
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result
```

## Proposed Solution

To solve this, we need dialect-aware serialization throughout the AST hierarchy:

Add `to_sql(&dyn Dialect)` to all AST types
- Add `to_sql(&dyn Dialect) -> String` method to `Statement`, `Query`, `Expr`, `ColumnDef`, and other AST types
- Each type's implementation calls `to_sql()` on its children, propagating the dialect
- Keep existing `Display` implementations unchanged for backwards compatibility

```rust
// Example usage after fix:
let ast = Parser::parse_sql(&dialect, sql).unwrap();
let regenerated = ast[0].to_sql(&dialect); // Correct PascalCase for ClickHouse
```

### Implementation Scope

The affected types include (non-exhaustive):
- `Statement` (top-level)
- `Query`, `SetExpr`, `Select`
- `Expr` (especially `Cast`, `TryCast`, `SafeCast`)
- `ColumnDef`, `ColumnOption`
- `TableConstraint`
- `AlterTableOperation`
- `FunctionArg`, `FunctionArgExpr`

This is a significant change but provides the cleanest API and maintains backwards compatibility.

## Workarounds

Currently, users must post-process the SQL string output to fix casing. See [514-labs/moosestack#3152](https://github.com/514-labs/moosestack/pull/3152) for an example regex-based workaround.

## References

- [ClickHouse Data Types Documentation](https://clickhouse.com/docs/en/sql-reference/data-types)
- Related workaround: [514-labs/moosestack#3152](https://github.com/514-labs/moosestack/pull/3152)

Guía de contribución

No hay ninguna guía de contribución indexada para este repositorio

Línea de trabajo

Comienza inspeccionando la implementación de DataType Display y los tipos AST enumerados en el issue, incluidos Statement, Query, Expr, ColumnDef y los nodos relacionados. Traza cómo la serialización llega a los tipos de datos anidados y, después, evalúa la API to_sql propuesta, consciente del dialecto, a través de toda la jerarquía. Se considera terminado cuando el ejemplo de ClickHouse hace round-trip con tipos PascalCase y el comportamiento existente de Display sigue siendo compatible hacia atrás.

Escrito por el modelo de indexación a partir del texto del issue.

Evaluación

Stack tecnológico
clickhouse, rust
Área
databases
Tipo de issue
Error
Dificultad
5/5
Tiempo estimado
Más de una semana
Estado de actividad
Estancado
Claridad
Bastante claro
Aptitud para principiantes
35/100

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.