ClickHouse / ClickHouse/clickhouse-cpp
Reading LowCardinality(Nullable(String)) column is not supported (for the data received from ClickHouse)
Nobody has claimed this yet.
- Dominant language
- C
- Stars
- 382
- Forks
- 208
- Avg merge
- 2d 19h
- Merged PRs (30d)
- 14
Description
Please see https://github.com/ClickHouse/clickhouse-cpp/pull/339
This one confirms that it works
Perhaps, it would be helpful to know the real type of column at run-time with:
column->GetType().GetName()
Originally posted by @Enmk in #326
The test Map_LowCardinalityTString_LowCardinalityTNullableString might succeed, but this is only true because you create the column with the type ColumnLowCardinalityT<ColumnNullableT<ColumnString>> explicitly in the test.
However, for the real data received from ClickHouse the type of the underlying column will be simply ColumnLowCardinality. This is probably due to the logic implemented in https://github.com/ClickHouse/clickhouse-cpp/pull/182:
case Type::Nullable:
return std::make_shared<ColumnLowCardinality>(
std::make_shared<ColumnNullable>(
CreateColumnFromAst(GetASTChildElement(nested, 0), settings),
std::make_shared<ColumnUInt8>()
)
);
Therefore the dynamic_cast inside column->As<ColumnLowCardinalityT<ColumnNullableT<ColumnString>>>() will yield nullptr.
This is easily illustrated with a simple typeid check (assume that column is an instance of a ColumnRef pointer).
This is what we have for LowCardinality(String) column:
// ColumnRef column;
auto type_name = column->GetType().GetName(); // "LowCardinality(String)"
auto type_id = typeid(*column).name(); // clickhouse::ColumnLowCardinalityT<clickhouse::ColumnString> ("N10clickhouse21ColumnLowCardinalityTINS_12ColumnStringEEE")
And this is the result for LowCardinality(Nullable(String)):
// ColumnRef column;
auto type_name = column->GetType().GetName(); // "LowCardinality(Nullable(String))"
auto type_id = typeid(*column).name(); // clickhouse::ColumnLowCardinality ("N10clickhouse20ColumnLowCardinalityE")
The conclusion is that it's not possible to read LowCardinality(Nullable(String)) column in the current version of clickhouse-cpp (v2.6.1):
column->As<ColumnLowCardinalityT<ColumnNullableT<ColumnString>>>()doesn't work for the reasons explained above.- While
column->As<ColumnLowCardinality>()does work, it's of no help: trying to wrap the result intoColumnLowCardinalityT<ColumnNullableT<ColumnString>>will also fail because the nested type isColumnNullable(notColumnNullableT<ColumnString>). - Declaring
ColumnLowCardinalityT<ColumnNullable>is not possible because there's noValueTypealias inColumnNullable.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with clickhouse/columns/factory.cpp around the Nullable handling and compare it with ut/roundtrip_tests.cpp at Map_LowCardinalityTString_LowCardinalityTNullableString_. Inspect ColumnRef::As and GetType().GetName() for columns received from ClickHouse, rather than only locally constructed columns. Done means LowCardinality(Nullable(String)) data can be read from a real ClickHouse response and has a regression test.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100