ClickHouse / ClickHouse/clickhouse-cs

ClickHouseDataReader silently returns zero rows when every column is zero-width (Tuple())

Open
#570 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
C#
Stars
94
Forks
22
Avg merge
11h 26m
Merged PRs (30d)
22

Description

### Describe the bug

`ClickHouseDataReader.Read()` decides it has reached the end of the result set with the byte-presence probe `reader.PeekChar() == -1` (`ClickHouse.Driver/ADO/Readers/ClickHouseDataReader.cs:705`, and the same probe in `TryMaterializeNextRow` at line 633).

`Tuple()` occupies **zero bytes** in RowBinary. A result set in which *every* column is zero-width therefore has zero-byte rows, and the probe cannot tell those rows from the end of the stream. Every such row is dropped **silently** — no exception, no warning, `Read()` simply returns `false` on the first call.

This is silent under-reporting of rows, which is worse than an error: a caller that iterates the reader sees an empty result and cannot detect that data was lost.

Zero-width column types observed on server 26.7.3.19: `Tuple()` and `Tuple(Tuple())`. Other "empty" types are *not* zero-width and are unaffected (`Array(Nothing)`, `Map(Nothing, Nothing)` and `Nullable(Nothing)` all write 1 byte per row).

### Steps to reproduce

1. Run any query whose columns are all `Tuple()`, e.g. `SELECT tuple()`.
2. Read it with `ExecuteReaderAsync` (or `ExecuteScalarAsync`).
3. `Read()` returns `false` on the first call; `ExecuteScalarAsync` returns `null`.

Note: reaching this code currently needs the `Tuple()` type-parser fix from #567 / PR #569, otherwise the type parse crashes first. The two defects are independent.

### Expected behaviour

The reader returns the same number of rows the server sent.

Server evidence (same devbox stack, ClickHouse 26.7.3.19):

```
$ curl -s 'http://clickhouse:8123/?query=SELECT tuple() FORMAT JSONCompact'
... "data": [[[]]], "rows": 1 ...

$ curl -s 'http://clickhouse:8123/?query=SELECT count() FROM (SELECT tuple())'
1

$ curl -s 'http://clickhouse:8123/?query=SELECT tuple() FROM numbers(3) FORMAT RowBinary' | wc -c
0 # 3 rows, 0 bytes of row data
```

### Code example

```csharp
using var conn = new ClickHouseConnection(connectionString);
await conn.OpenAsync();

using var cmd = conn.CreateCommand();
cmd.CommandText = "SELECT tuple() FROM numbers(3)";
using var reader = await cmd.ExecuteReaderAsync();

int rows = 0;
while (reader.Read()) rows++;
Console.WriteLine(rows); // prints 0; the server sent 3
```

Observed output of a sweep over the plausible shapes (driver at `1e5971c` plus the PR #569 parser fix, so the type parses):

```
SELECT tuple() => reader rows=0, fields=1, scalar=
SELECT tuple() FROM numbers(3) => reader rows=0, fields=1, scalar=
SELECT tuple(tuple()) => reader rows=0, fields=1, scalar=
SELECT tuple(), tuple() => reader rows=0, fields=2, scalar=
SELECT tuple(), 5 => reader rows=1, fields=2, scalar=LargeTuple <-- correct
SELECT 5 => reader rows=1, fields=1, scalar=5 <-- correct
```

The contrast case matters: as soon as one non-zero-width column is present, the rows have a non-zero byte length and the reader is correct. Only the all-zero-width case is affected.

### Error log

None — this is the point of the report. No exception is raised.

### Configuration

#### Environment
* Client version: `main` @ `1e5971c` (plus PR #569 applied locally so `Tuple()` parses)
* .NET version: 10.0
* OS: Ubuntu 24.04 (docker)

#### ClickHouse server
* ClickHouse Server version: 26.7.3.19
* Non-default settings: none
* `CREATE TABLE` statements: none needed — `SELECT tuple()` reproduces it. A stored column reproduces it too: `CREATE TABLE t (x Tuple()) ENGINE Memory`.

### Root cause

`ClickHouse.Driver/ADO/Readers/ClickHouseDataReader.cs:705` (`Read()`) and `:633` (`TryMaterializeNextRow`) infer "no more rows" from "no more bytes". That inference holds only while every row consumes at least one byte. With `Tuple()` columns the row width is 0, so "no more bytes" is true even at row 0 of N.

The row count genuinely cannot be recovered from the RowBinary byte stream in this case: the server's own `RowBinary` output for 3 zero-width rows is byte-identical to its output for 0 rows (0 bytes, shown above). So this is not a probe that can be repaired in place.

### Suggested fix

The row boundary has to come from somewhere other than byte presence. Options, in rough order of intrusiveness — this is likely a maintainer design decision:

1. **Detect the degenerate case and get the count out of band.** When every parsed column type has a fixed zero wire width, the reader knows the byte stream carries no row information at all, and can source the count from the response's `X-ClickHouse-Summary` header (`rows_read` / `read_rows`) or from a wrapping query. Narrow blast radius: no change to any normal result set.
2. **Reject rather than silently truncate.** If the count cannot be established, throw on a result set whose columns are all zero-width instead of reporting zero rows. Strictly worse for the user than option 1, but far better than silent data loss.
3. **Change the request format** for such result sets to one that frames rows explicitly (e.g. a `Native` block header carrying the row count). Largest change; affects the whole read path.

Contrast case that must keep working: any result set with at least one non-zero-width column (`SELECT tuple(), 5` above) already reads correctly and must not regress.

The same probe also guards `TryMaterializeNextRow` (the box-free POCO path), so both `ExecuteReaderAsync` and `QueryAsync` are affected and any fix should cover both.

Contributor guide

Open the contributing guide

Research direction

Start in ClickHouse.Driver/ADO/Readers/ClickHouseDataReader.cs at the probes in TryMaterializeNextRow around line 633 and Read() around line 705, then reproduce SELECT tuple() FROM numbers(3) and the mixed-width SELECT tuple(), 5 case. Trace how row boundaries and response metadata are available for both reader paths; done means zero-width result sets no longer silently report zero rows while mixed-width results remain correct.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
backend, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.