ClickHouse / ClickHouse/clickhouse-java
[client-v2, jdbc-v2] A caller-supplied FORMAT clause is silently ignored on ClickHouse 26.8+ (X-ClickHouse-Format wins)
- 主要言語
- Java
- スター
- 1.6k
- フォーク
- 636
- 平均マージ
- 2日 23時間
- マージ済み PR(30日)
- 29
説明
## Description
On ClickHouse 26.8+ the `X-ClickHouse-Format` request header is an explicit override that
wins over the query's `FORMAT` clause. client-v2 sends that header on **every** request, and
defaults it to `RowBinaryWithNamesAndTypes` when the caller sets no format. A caller who
writes `SELECT ... FORMAT JSONEachRow` therefore receives `RowBinaryWithNamesAndTypes` bytes
instead. There is no exception — only different bytes.
This contradicts the documented contract of `Client.query(...)`:
> Server response format can be specified through `settings` or in SQL query.
> If specified in both, the `sqlQuery` will take precedence.
>
> — `client-v2/src/main/java/com/clickhouse/client/api/Client.java` (Javadoc of `query`)
Related but different: issue #3068 covers the client's own **internal** metadata queries
(`getTableSchema`, `getTableSchemaFromQuery`, `ping`), which fail loudly. This report is the
**caller-facing** half, which fails silently. The two do not share a fix: #3068 is fixed by
changing the client's own SQL, this one is about what the client does with SQL the user wrote.
Server-side cause is the same: ClickHouse PR ClickHouse/ClickHouse#105249 turned
`X-ClickHouse-Format` from an alias of `default_format` (a fallback) into an alias of `format`
(an explicit override).
### Steps to reproduce
1. Start `clickhouse/clickhouse-server:head` (26.8.1.1761).
2. Run `client.query("SELECT 1 AS a, 'x' AS b FORMAT JSONEachRow", new QuerySettings())`.
3. Read `response.getFormat()` and the response body.
Three tests already in the repository pin this contract and turn red on 26.8, while passing
on 26.6.2.160 (same worktree, same build, only the server image changed):
| Test | Result on 26.8.1.1761 |
|---|---|
| `client-v2` `QueryTests.testSettingsNotChanged` | `expected [JSONEachRow] but found [RowBinaryWithNamesAndTypes]` |
| `jdbc-v2` `StatementTest.testJSONEachRowFormatRequiresParserFactory` | `Expected SQLException` (none thrown) |
| `jdbc-v2` `StatementTest.testTextFormatInResponse` | `Expected SQLException to be thrown, but nothing was thrown` |
`jdbc-v2` `StatementTest.testJSONEachRowFormat` still passes, but for the wrong reason: its
assertions (`getInt`, `getString`) hold for either format, so the silent switch to RowBinary
is invisible to it.
The two `jdbc-v2` failures show that the shipped, documented `FORMAT JSONEachRow` support in
the JDBC driver (`jdbc_json_parser_factory`, `docs/client-v2-json-support.md`) is inert on
26.8: `response.getFormat()` can no longer be `JSONEachRow`, so `StatementImpl` never selects
`JSONEachRowFormatReader` and returns binary-decoded rows instead.
### Error Log or Exception StackTrace
```
No exception is thrown. That is the problem.
client-v2 QueryTests.testSettingsNotChanged:2345
expected [JSONEachRow] but found [RowBinaryWithNamesAndTypes]
```
### Expected Behaviour
Either of the following, but not the current silent mismatch:
* the `FORMAT` clause keeps taking precedence, as `Client.query`'s Javadoc states; or
* the `FORMAT` clause is no longer supported and the client raises a clear error, with the
Javadoc, `docs/client-v2-json-support.md`, `docs/features.md` and the
`examples/jdbc-v2-json-processors` example updated accordingly.
Server behaviour, isolated with `curl` — same query, same header, two servers:
```
$ curl -H 'X-ClickHouse-Format: TabSeparated' ... --data-binary "SELECT 1 AS a FORMAT JSONEachRow"
26.6.2.160: {"a":1} <- FORMAT clause wins
26.8.1.1761: 1 <- header wins
```
With no header at all, 26.8.1.1761 returns `{"a":1}`, so `FORMAT JSONEachRow` itself is still
honoured by the server; only the client's unconditional header suppresses it.
### Code Example
```java
try (QueryResponse response =
client.query("SELECT 1 AS a, 'x' AS b FORMAT JSONEachRow", new QuerySettings()).get()) {
System.out.println(response.getFormat());
// read response.getInputStream()
}
```
Observed against 26.8.1.1761:
```
response.getFormat() = RowBinaryWithNamesAndTypes
raw bytes (hex) = 02 01 61 01 62 05 55 49 6e 74 38 06 53 74 72 69 6e 67 01 01 78
```
That is `RowBinaryWithNamesAndTypes` (2 columns; `a`/`b`; `UInt8`/`String`; values 1, "x").
Expected `{"a":1,"b":"x"}`.
## Root cause
* `client-v2/src/main/java/com/clickhouse/client/api/Client.java`, `query(...)` — when the
caller sets no format, the request settings are defaulted:
`requestSettings.setFormat(ClickHouseFormat.RowBinaryWithNamesAndTypes)`. The SQL text is
never inspected for a `FORMAT` clause.
* `client-v2/src/main/java/com/clickhouse/client/api/internal/HttpAPIClientHelper.java`,
`addHeaders(...)` — that setting is emitted unconditionally as
`X-ClickHouse-Format` (`ClickHouseHttpProto.HEADER_FORMAT`).
Before 26.8 the two disagreeing instructions were harmless because the clause won. From 26.8
the header wins, so the default silently overrides the caller.
`jdbc-v2` inherits this: `StatementImpl.executeQueryImpl` branches on `response.getFormat()`,
which is read back from the response header, so its `JSONEachRow` and text-format branches
have become unreachable.
## Suggested fix
Make the header and the SQL agree, whichever direction the project prefers:
1. **Keep honouring the clause.** `ClickHouseSqlStatement.getFormat()` already extracts a
`FORMAT` clause; feed it into the request settings so the header matches what the caller
asked for. This preserves the Javadoc contract, the existing tests, and the `jdbc-v2` JSON
support, and needs no user-visible change.
2. **Deprecate the clause.** Detect a `FORMAT` clause in the caller's SQL and fail with a
clear error pointing at `QuerySettings.setFormat(...)`. This matches the direction stated
on #3068 ("format should be sent via settings not in query"), but it is a breaking change
and requires updating the Javadoc, the three tests above, `docs/client-v2-json-support.md`,
`docs/features.md` and `examples/jdbc-v2-json-processors`.
What must not remain is the current state, where the client accepts the clause and quietly
returns a different format.
Contrast case that must keep working: a caller who sets the format only through
`QuerySettings.setFormat(...)` and writes no `FORMAT` clause is unaffected today and must stay
unaffected. Inserts and client-v1 already keep header and clause in agreement; r2dbc sends no
format header.
## Configuration
#### Client Configuration
```java
Client client = new Client.Builder()
.addEndpoint(Protocol.HTTP, host, 8123, false)
.setUsername("default").setPassword(password)
.build();
```
#### Environment
* Client version: 0.10.0-rc1-SNAPSHOT, built from `main` at 40464dde1
* Language version: Java 17
* OS: Ubuntu 24.04 (Docker)
#### ClickHouse Server
* ClickHouse Server version: 26.8.1.1761 (`clickhouse/clickhouse-server:head`) reproduces;
26.6.2.160 does not
* ClickHouse Server non-default settings, if any: none
* `CREATE TABLE` statements for tables involved: none, `SELECT 1 AS a, 'x' AS b` is enough
* Sample data for all these tables: none needed
---
Found by automated analysis of clickhouse-java while working on #3068, and verified against a
live 26.8.1.1761 server and a 26.6.2.160 control rather than by inspection.
コントリビューションガイド
調査の方向性
client-v2/src/main/java/com/clickhouse/client/api/Client.java、HttpAPIClientHelper.java、ClickHouseSqlStatement.getFormat() から始め、指定されたサーバーバージョンに対して、名前が示されている client-v2 と jdbc-v2 の 3 つのテストを実行します。完了の条件は、プロジェクトが呼び出し元から指定された FORMAT 句に対する明示的な動作を 1 つ選択して文書化し、設定のみのフォーマットを維持し、対象のテストとドキュメントにあるサイレントな不一致を取り除くことです。
索引モデルが issue の本文から書いたものです。
評価
- 技術スタック
- java, sql
- 領域
- api, databases
- issue の種類
- バグ
- 難易度
- 4/5
- 見積もり時間
- 3〜5日
- 活発さ
- 活発
- 明瞭さ
- おおむね明確
- 初心者へのやさしさ
- 48/100