apache / apache/datafusion-sqlparser-rs

ClickHouse: `Display` implementation converts types to uppercase, causing `UNKNOWN_TYPE` errors

Ouverte
#2,153 0 commentaires 0 réactions 0 personnes assignées Voir sur GitHub
Langage dominant
Rust
Étoiles
3.5k
Forks
772
Merge moyen
4 j 9 h
PR mergées (30 j)
17

Description

## Problem

ClickHouse data types are case-sensitive and require PascalCase (e.g., `String`, `Int32`, `Nullable`). However, the `sqlparser-rs` library's `Display` implementation for `DataType` converts certain types to uppercase, causing `UNKNOWN_TYPE` errors when round-tripping SQL through ClickHouse.

## Example
```rust
use sqlparser::dialect::ClickHouseDialect;
use sqlparser::parser::Parser;

let sql = "CREATE TABLE t (col Nullable(String))";
let dialect = ClickHouseDialect {};
let ast = Parser::parse_sql(&dialect, sql).unwrap();

// Round-trip: parse and convert back to string
let regenerated = ast[0].to_string();
// Result: "CREATE TABLE t (col Nullable(STRING))"
// ^^^^^^ uppercase!
```

When this regenerated SQL is executed against ClickHouse, it fails with:

```
Code: 47. DB::Exception: Unknown type STRING. (UNKNOWN_TYPE)
```

## Affected Types

| Type | Current Output | ClickHouse Requires |
|------|----------------|---------------------|
| `DataType::Int8` | `INT8` | `Int8` |
| `DataType::Int64` | `INT64` | `Int64` |
| `DataType::Float64` | `FLOAT64` | `Float64` |
| `DataType::String` | `STRING` | `String` |
| `DataType::Bool` | `BOOL` | `Bool` |
| `DataType::Date` | `DATE` | `Date` |
| `DataType::Datetime` | `DATETIME` | `DateTime` |

**Types already correct (PascalCase):**
- `Int16`, `Int32`, `Int128`, `Int256`
- `UInt8`, `UInt16`, `UInt32`, `UInt64`, `UInt128`, `UInt256`
- `Float32`
- `Nullable`, `LowCardinality`, `Array`, `Map`, `Tuple`, `Nested`

## Root Cause

The `Display` trait implementation for `DataType` uses uppercase for type names (e.g., `write!(f, "STRING")`), which is standard for most SQL dialects but incorrect for ClickHouse.

The challenge is that `Display` doesn't have access to dialect context, so it can't conditionally format based on the active dialect.

## The Problem with `Display`

Most users serialize SQL by calling `Display` on top-level AST types:

```rust
let ast = Parser::parse_sql(&dialect, sql).unwrap();
let regenerated = ast[0].to_string(); // Uses Display on Statement
// or
let regenerated = format!("{}", query); // Uses Display on Query
```

The `Display` trait signature doesn't allow passing context:
```rust
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result
```

## Proposed Solution

To solve this, we need dialect-aware serialization throughout the AST hierarchy:

Add `to_sql(&dyn Dialect)` to all AST types
- Add `to_sql(&dyn Dialect) -> String` method to `Statement`, `Query`, `Expr`, `ColumnDef`, and other AST types
- Each type's implementation calls `to_sql()` on its children, propagating the dialect
- Keep existing `Display` implementations unchanged for backwards compatibility

```rust
// Example usage after fix:
let ast = Parser::parse_sql(&dialect, sql).unwrap();
let regenerated = ast[0].to_sql(&dialect); // Correct PascalCase for ClickHouse
```

### Implementation Scope

The affected types include (non-exhaustive):
- `Statement` (top-level)
- `Query`, `SetExpr`, `Select`
- `Expr` (especially `Cast`, `TryCast`, `SafeCast`)
- `ColumnDef`, `ColumnOption`
- `TableConstraint`
- `AlterTableOperation`
- `FunctionArg`, `FunctionArgExpr`

This is a significant change but provides the cleanest API and maintains backwards compatibility.

## Workarounds

Currently, users must post-process the SQL string output to fix casing. See [514-labs/moosestack#3152](https://github.com/514-labs/moosestack/pull/3152) for an example regex-based workaround.

## References

- [ClickHouse Data Types Documentation](https://clickhouse.com/docs/en/sql-reference/data-types)
- Related workaround: [514-labs/moosestack#3152](https://github.com/514-labs/moosestack/pull/3152)

Guide de contribution

Aucun guide de contribution indexé pour ce dépôt

Piste de recherche

Commencez par inspecter l’implémentation de DataType Display et les types AST listés dans l’issue, notamment Statement, Query, Expr, ColumnDef et les nœuds associés. Suivez le cheminement de la sérialisation jusqu’aux types de données imbriqués, puis évaluez l’API to_sql proposée, sensible au dialecte, dans l’ensemble de la hiérarchie. Le travail est considéré comme terminé lorsque l’exemple ClickHouse effectue un aller-retour avec des types PascalCase et que le comportement existant de Display reste rétrocompatible.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Évaluation

Stack technique
clickhouse, rust
Domaine
databases
Type d'issue
Bug
Difficulté
5/5
Temps estimé
Plus d'une semaine
Activité
À l'abandon
Clarté
Plutôt claire
Accessibilité débutants
35/100

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.