ClickHouse / ClickHouse/clickhouse-java

[client-v2, jdbc-v2] A caller-supplied FORMAT clause is silently ignored on ClickHouse 26.8+ (X-ClickHouse-Format wins)

未关闭
#3,070 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
area:format bug client-api-v2 jdbc-v2
主要语言
Java
星标
1.6k
派生
636
平均合并
2 天 23 小时
30 天内合并 PR
29

描述

## Description

On ClickHouse 26.8+ the `X-ClickHouse-Format` request header is an explicit override that
wins over the query's `FORMAT` clause. client-v2 sends that header on **every** request, and
defaults it to `RowBinaryWithNamesAndTypes` when the caller sets no format. A caller who
writes `SELECT ... FORMAT JSONEachRow` therefore receives `RowBinaryWithNamesAndTypes` bytes
instead. There is no exception — only different bytes.

This contradicts the documented contract of `Client.query(...)`:

> Server response format can be specified through `settings` or in SQL query.
> If specified in both, the `sqlQuery` will take precedence.
>
> — `client-v2/src/main/java/com/clickhouse/client/api/Client.java` (Javadoc of `query`)

Related but different: issue #3068 covers the client's own **internal** metadata queries
(`getTableSchema`, `getTableSchemaFromQuery`, `ping`), which fail loudly. This report is the
**caller-facing** half, which fails silently. The two do not share a fix: #3068 is fixed by
changing the client's own SQL, this one is about what the client does with SQL the user wrote.

Server-side cause is the same: ClickHouse PR ClickHouse/ClickHouse#105249 turned
`X-ClickHouse-Format` from an alias of `default_format` (a fallback) into an alias of `format`
(an explicit override).

### Steps to reproduce

1. Start `clickhouse/clickhouse-server:head` (26.8.1.1761).
2. Run `client.query("SELECT 1 AS a, 'x' AS b FORMAT JSONEachRow", new QuerySettings())`.
3. Read `response.getFormat()` and the response body.

Three tests already in the repository pin this contract and turn red on 26.8, while passing
on 26.6.2.160 (same worktree, same build, only the server image changed):

| Test | Result on 26.8.1.1761 |
|---|---|
| `client-v2` `QueryTests.testSettingsNotChanged` | `expected [JSONEachRow] but found [RowBinaryWithNamesAndTypes]` |
| `jdbc-v2` `StatementTest.testJSONEachRowFormatRequiresParserFactory` | `Expected SQLException` (none thrown) |
| `jdbc-v2` `StatementTest.testTextFormatInResponse` | `Expected SQLException to be thrown, but nothing was thrown` |

`jdbc-v2` `StatementTest.testJSONEachRowFormat` still passes, but for the wrong reason: its
assertions (`getInt`, `getString`) hold for either format, so the silent switch to RowBinary
is invisible to it.

The two `jdbc-v2` failures show that the shipped, documented `FORMAT JSONEachRow` support in
the JDBC driver (`jdbc_json_parser_factory`, `docs/client-v2-json-support.md`) is inert on
26.8: `response.getFormat()` can no longer be `JSONEachRow`, so `StatementImpl` never selects
`JSONEachRowFormatReader` and returns binary-decoded rows instead.

### Error Log or Exception StackTrace

```
No exception is thrown. That is the problem.

client-v2 QueryTests.testSettingsNotChanged:2345
expected [JSONEachRow] but found [RowBinaryWithNamesAndTypes]
```

### Expected Behaviour

Either of the following, but not the current silent mismatch:

* the `FORMAT` clause keeps taking precedence, as `Client.query`'s Javadoc states; or
* the `FORMAT` clause is no longer supported and the client raises a clear error, with the
Javadoc, `docs/client-v2-json-support.md`, `docs/features.md` and the
`examples/jdbc-v2-json-processors` example updated accordingly.

Server behaviour, isolated with `curl` — same query, same header, two servers:

```
$ curl -H 'X-ClickHouse-Format: TabSeparated' ... --data-binary "SELECT 1 AS a FORMAT JSONEachRow"
26.6.2.160: {"a":1} <- FORMAT clause wins
26.8.1.1761: 1 <- header wins
```

With no header at all, 26.8.1.1761 returns `{"a":1}`, so `FORMAT JSONEachRow` itself is still
honoured by the server; only the client's unconditional header suppresses it.

### Code Example

```java
try (QueryResponse response =
client.query("SELECT 1 AS a, 'x' AS b FORMAT JSONEachRow", new QuerySettings()).get()) {
System.out.println(response.getFormat());
// read response.getInputStream()
}
```

Observed against 26.8.1.1761:

```
response.getFormat() = RowBinaryWithNamesAndTypes
raw bytes (hex) = 02 01 61 01 62 05 55 49 6e 74 38 06 53 74 72 69 6e 67 01 01 78
```

That is `RowBinaryWithNamesAndTypes` (2 columns; `a`/`b`; `UInt8`/`String`; values 1, "x").
Expected `{"a":1,"b":"x"}`.

## Root cause

* `client-v2/src/main/java/com/clickhouse/client/api/Client.java`, `query(...)` — when the
caller sets no format, the request settings are defaulted:
`requestSettings.setFormat(ClickHouseFormat.RowBinaryWithNamesAndTypes)`. The SQL text is
never inspected for a `FORMAT` clause.
* `client-v2/src/main/java/com/clickhouse/client/api/internal/HttpAPIClientHelper.java`,
`addHeaders(...)` — that setting is emitted unconditionally as
`X-ClickHouse-Format` (`ClickHouseHttpProto.HEADER_FORMAT`).

Before 26.8 the two disagreeing instructions were harmless because the clause won. From 26.8
the header wins, so the default silently overrides the caller.

`jdbc-v2` inherits this: `StatementImpl.executeQueryImpl` branches on `response.getFormat()`,
which is read back from the response header, so its `JSONEachRow` and text-format branches
have become unreachable.

## Suggested fix

Make the header and the SQL agree, whichever direction the project prefers:

1. **Keep honouring the clause.** `ClickHouseSqlStatement.getFormat()` already extracts a
`FORMAT` clause; feed it into the request settings so the header matches what the caller
asked for. This preserves the Javadoc contract, the existing tests, and the `jdbc-v2` JSON
support, and needs no user-visible change.
2. **Deprecate the clause.** Detect a `FORMAT` clause in the caller's SQL and fail with a
clear error pointing at `QuerySettings.setFormat(...)`. This matches the direction stated
on #3068 ("format should be sent via settings not in query"), but it is a breaking change
and requires updating the Javadoc, the three tests above, `docs/client-v2-json-support.md`,
`docs/features.md` and `examples/jdbc-v2-json-processors`.

What must not remain is the current state, where the client accepts the clause and quietly
returns a different format.

Contrast case that must keep working: a caller who sets the format only through
`QuerySettings.setFormat(...)` and writes no `FORMAT` clause is unaffected today and must stay
unaffected. Inserts and client-v1 already keep header and clause in agreement; r2dbc sends no
format header.

## Configuration

#### Client Configuration
```java
Client client = new Client.Builder()
.addEndpoint(Protocol.HTTP, host, 8123, false)
.setUsername("default").setPassword(password)
.build();
```

#### Environment
* Client version: 0.10.0-rc1-SNAPSHOT, built from `main` at 40464dde1
* Language version: Java 17
* OS: Ubuntu 24.04 (Docker)

#### ClickHouse Server
* ClickHouse Server version: 26.8.1.1761 (`clickhouse/clickhouse-server:head`) reproduces;
26.6.2.160 does not
* ClickHouse Server non-default settings, if any: none
* `CREATE TABLE` statements for tables involved: none, `SELECT 1 AS a, 'x' AS b` is enough
* Sample data for all these tables: none needed

---

Found by automated analysis of clickhouse-java while working on #3068, and verified against a
live 26.8.1.1761 server and a 26.6.2.160 control rather than by inspection.

贡献指南

打开贡献指南

调研方向

从 client-v2/src/main/java/com/clickhouse/client/api/Client.java、HttpAPIClientHelper.java 和 ClickHouseSqlStatement.getFormat() 开始,然后针对所述的服务器版本运行列出的三个 client-v2 和 jdbc-v2 测试。完成的标准是:项目为调用方提供的 FORMAT 子句选择并记录一种明确的行为,保留仅包含设置的格式,并消除受影响测试和文档中的静默不匹配。

由索引模型根据 Issue 内容生成。

评估

技术栈
java, sql
领域
api, databases
Issue 类型
缺陷
难度
4/5
预计耗时
3-5 天
活跃度
活跃
描述清晰度
基本清楚
新手友好度
48/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。