ClickHouse / ClickHouse/clickhouse-cpp

Reading LowCardinality(Nullable(String)) column is not supported (for the data received from ClickHouse)

Offen
#504 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
Vorherrschende Sprache
C
Sterne
382
Forks
209
Ø Merge
1 T. 22 Std.
Gemergte PRs (30 T.)
13

Beschreibung

>
> Please see https://github.com/ClickHouse/clickhouse-cpp/pull/339
>
>
>
> [This one confirms that it works](https://github.com/ClickHouse/clickhouse-cpp/pull/339/files#diff-5ba05c8bd7182b90fb7e5b5b010e00c29c723612eb084a0b24b6f897fb45c531L215)
>
>
>
> Perhaps, it would be helpful to know the real type of column at run-time with:
>
> ```c++
>
> column->GetType().GetName()
>
> ```

_Originally posted by @Enmk in [#326](https://github.com/ClickHouse/clickhouse-cpp/issues/326#issuecomment-1759450216)_

The [test](https://github.com/ClickHouse/clickhouse-cpp/blob/v2.6.1/ut/roundtrip_tests.cpp#L279) _Map_LowCardinalityTString_LowCardinalityTNullableString_ might succeed, but this is only true because you create the column with the type `ColumnLowCardinalityT>` explicitly in the test.

However, for the real data received from _ClickHouse_ the type of the underlying column will be simply `ColumnLowCardinality`. This is probably due to the [logic](https://github.com/ClickHouse/clickhouse-cpp/blob/v2.6.1/clickhouse/columns/factory.cpp#L226) implemented in https://github.com/ClickHouse/clickhouse-cpp/pull/182:
```
case Type::Nullable:
return std::make_shared(
std::make_shared(
CreateColumnFromAst(GetASTChildElement(nested, 0), settings),
std::make_shared()
)
);
```
Therefore the `dynamic_cast` inside `column->As>>()` will yield _nullptr_.

This is easily illustrated with a simple **typeid** check (assume that `column` is an instance of a `ColumnRef` pointer).

This is what we have for `LowCardinality(String)` column:
```
// ColumnRef column;

auto type_name = column->GetType().GetName(); // "LowCardinality(String)"

auto type_id = typeid(*column).name(); // clickhouse::ColumnLowCardinalityT ("N10clickhouse21ColumnLowCardinalityTINS_12ColumnStringEEE")
```
And this is the result for `LowCardinality(Nullable(String))`:
```
// ColumnRef column;

auto type_name = column->GetType().GetName(); // "LowCardinality(Nullable(String))"

auto type_id = typeid(*column).name(); // clickhouse::ColumnLowCardinality ("N10clickhouse20ColumnLowCardinalityE")
```

The conclusion is that it's not possible to read `LowCardinality(Nullable(String))` column in the current version of _clickhouse-cpp_ (_v2.6.1_):
1. `column->As>>()` doesn't work for the reasons explained above.
2. While `column->As()` does work, it's of no help: trying to wrap the result into `ColumnLowCardinalityT>` will also fail because the nested type is `ColumnNullable` (not `ColumnNullableT`).
3. Declaring `ColumnLowCardinalityT` is not possible because there's no `ValueType` alias in `ColumnNullable`.

Beitragsleitfaden

Für dieses Repository ist kein Beitragsleitfaden indexiert

Rechercherichtung

Beginne in clickhouse/columns/factory.cpp bei der Nullable-Behandlung und vergleiche sie mit ut/roundtrip_tests.cpp bei Map_LowCardinalityTString_LowCardinalityTNullableString_. Untersuche ColumnRef::As und GetType().GetName() für Spalten, die von ClickHouse empfangen werden, statt nur lokal erzeugte Spalten zu betrachten. Die Aufgabe ist erledigt, wenn Daten vom Typ LowCardinality(Nullable(String)) aus einer echten ClickHouse-Antwort gelesen werden können und es einen Regressionstest gibt.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Bewertung

Tech-Stack
cpp
Bereich
databases
Issue-Typ
Bug
Schwierigkeit
4/5
Geschätzter Aufwand
3-5 Tage
Aktivitätsstatus
Ruhig
Klarheit
Größtenteils klar
Anfängerfreundlichkeit
48/100

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.