ClickHouse / ClickHouse/clickhouse-cs
ClickHouseDataReader silently returns zero rows when every column is zero-width (Tuple())
- Dominant language
- C#
- Stars
- 94
- Forks
- 22
- Avg merge
- 11h 26m
- Merged PRs (30d)
- 22
Description
### Describe the bug
`ClickHouseDataReader.Read()` decides it has reached the end of the result set with the byte-presence probe `reader.PeekChar() == -1` (`ClickHouse.Driver/ADO/Readers/ClickHouseDataReader.cs:705`, and the same probe in `TryMaterializeNextRow` at line 633).
`Tuple()` occupies **zero bytes** in RowBinary. A result set in which *every* column is zero-width therefore has zero-byte rows, and the probe cannot tell those rows from the end of the stream. Every such row is dropped **silently** — no exception, no warning, `Read()` simply returns `false` on the first call.
This is silent under-reporting of rows, which is worse than an error: a caller that iterates the reader sees an empty result and cannot detect that data was lost.
Zero-width column types observed on server 26.7.3.19: `Tuple()` and `Tuple(Tuple())`. Other "empty" types are *not* zero-width and are unaffected (`Array(Nothing)`, `Map(Nothing, Nothing)` and `Nullable(Nothing)` all write 1 byte per row).
### Steps to reproduce
1. Run any query whose columns are all `Tuple()`, e.g. `SELECT tuple()`.
2. Read it with `ExecuteReaderAsync` (or `ExecuteScalarAsync`).
3. `Read()` returns `false` on the first call; `ExecuteScalarAsync` returns `null`.
Note: reaching this code currently needs the `Tuple()` type-parser fix from #567 / PR #569, otherwise the type parse crashes first. The two defects are independent.
### Expected behaviour
The reader returns the same number of rows the server sent.
Server evidence (same devbox stack, ClickHouse 26.7.3.19):
```
$ curl -s 'http://clickhouse:8123/?query=SELECT tuple() FORMAT JSONCompact'
... "data": [[[]]], "rows": 1 ...
$ curl -s 'http://clickhouse:8123/?query=SELECT count() FROM (SELECT tuple())'
1
$ curl -s 'http://clickhouse:8123/?query=SELECT tuple() FROM numbers(3) FORMAT RowBinary' | wc -c
0 # 3 rows, 0 bytes of row data
```
### Code example
```csharp
using var conn = new ClickHouseConnection(connectionString);
await conn.OpenAsync();
using var cmd = conn.CreateCommand();
cmd.CommandText = "SELECT tuple() FROM numbers(3)";
using var reader = await cmd.ExecuteReaderAsync();
int rows = 0;
while (reader.Read()) rows++;
Console.WriteLine(rows); // prints 0; the server sent 3
```
Observed output of a sweep over the plausible shapes (driver at `1e5971c` plus the PR #569 parser fix, so the type parses):
```
SELECT tuple() => reader rows=0, fields=1, scalar=
SELECT tuple() FROM numbers(3) => reader rows=0, fields=1, scalar=
SELECT tuple(tuple()) => reader rows=0, fields=1, scalar=
SELECT tuple(), tuple() => reader rows=0, fields=2, scalar=
SELECT tuple(), 5 => reader rows=1, fields=2, scalar=LargeTuple <-- correct
SELECT 5 => reader rows=1, fields=1, scalar=5 <-- correct
```
The contrast case matters: as soon as one non-zero-width column is present, the rows have a non-zero byte length and the reader is correct. Only the all-zero-width case is affected.
### Error log
None — this is the point of the report. No exception is raised.
### Configuration
#### Environment
* Client version: `main` @ `1e5971c` (plus PR #569 applied locally so `Tuple()` parses)
* .NET version: 10.0
* OS: Ubuntu 24.04 (docker)
#### ClickHouse server
* ClickHouse Server version: 26.7.3.19
* Non-default settings: none
* `CREATE TABLE` statements: none needed — `SELECT tuple()` reproduces it. A stored column reproduces it too: `CREATE TABLE t (x Tuple()) ENGINE Memory`.
### Root cause
`ClickHouse.Driver/ADO/Readers/ClickHouseDataReader.cs:705` (`Read()`) and `:633` (`TryMaterializeNextRow`) infer "no more rows" from "no more bytes". That inference holds only while every row consumes at least one byte. With `Tuple()` columns the row width is 0, so "no more bytes" is true even at row 0 of N.
The row count genuinely cannot be recovered from the RowBinary byte stream in this case: the server's own `RowBinary` output for 3 zero-width rows is byte-identical to its output for 0 rows (0 bytes, shown above). So this is not a probe that can be repaired in place.
### Suggested fix
The row boundary has to come from somewhere other than byte presence. Options, in rough order of intrusiveness — this is likely a maintainer design decision:
1. **Detect the degenerate case and get the count out of band.** When every parsed column type has a fixed zero wire width, the reader knows the byte stream carries no row information at all, and can source the count from the response's `X-ClickHouse-Summary` header (`rows_read` / `read_rows`) or from a wrapping query. Narrow blast radius: no change to any normal result set.
2. **Reject rather than silently truncate.** If the count cannot be established, throw on a result set whose columns are all zero-width instead of reporting zero rows. Strictly worse for the user than option 1, but far better than silent data loss.
3. **Change the request format** for such result sets to one that frames rows explicitly (e.g. a `Native` block header carrying the row count). Largest change; affects the whole read path.
Contrast case that must keep working: any result set with at least one non-zero-width column (`SELECT tuple(), 5` above) already reads correctly and must not regress.
The same probe also guards `TryMaterializeNextRow` (the box-free POCO path), so both `ExecuteReaderAsync` and `QueryAsync` are affected and any fix should cover both.
Contributor guide
Research direction
Start in ClickHouse.Driver/ADO/Readers/ClickHouseDataReader.cs at the probes in TryMaterializeNextRow around line 633 and Read() around line 705, then reproduce SELECT tuple() FROM numbers(3) and the mixed-width SELECT tuple(), 5 case. Trace how row boundaries and response metadata are available for both reader paths; done means zero-width result sets no longer silently report zero rows while mixed-width results remain correct.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100