ClickHouse / ClickHouse/clickhouse-java

client-v1 (clickhouse-data): "Duplicate key" IllegalStateException on result sets with duplicate column names (0.9.8)

Open
#2,895 1 comment 0 reactions 0 assignees View on GitHub
bug client-v1 wontfix
Dominant language
Java
Stars
1.6k
Forks
636
Avg merge
2d 16h
Merged PRs (30d)
28

Description

## Description

The v1 client (`clickhouse-client` / `clickhouse-http-client` **0.9.8**) throws `java.lang.IllegalStateException: Duplicate key ...` when a query returns a result set with **duplicate column names** (which is valid SQL).

`com.clickhouse.data.ClickHouseDataProcessor$DefaultSerDe` builds a *column-name → index* map with `Collectors.toMap` (no merge function) at construction time, so any repeated column name fails immediately — **before the caller can read anything, even when reading purely by position**.

This reproduces with `RowBinaryWithNamesAndTypes` (the format that carries column names). A trivial `SELECT key1, key1 FROM (SELECT 1 AS key1)` is enough. Real-world trigger: a reporting tool that lets a user select the same projection twice (e.g. two `Date` columns), producing SQL like `SELECT toDate(...) AS Date, toDate(...) AS Date, ...`.

This looks related to #2336 (reported on 0.8.4, marked fixed in 0.9.3 via #2618), but that fix was in the JDBC `TableSchema` (Guava `ImmutableMap`, *"Multiple entries with same key"*). The failure here is a **different code path** — `ClickHouseDataProcessor` using the JDK `Collectors.toMap` (*"attempted merging values"*) — and **still fails in 0.9.8**.

### Steps to reproduce
1. Use `com.clickhouse:clickhouse-http-client:0.9.8` (v1 client).
2. Execute a query whose result has two columns with the same name, e.g. `SELECT key1, key1 FROM (SELECT 1 AS key1)`, with format `RowBinaryWithNamesAndTypes`.
3. `executeAndWait()` / iterating `response.records()` throws.

### Error Log or Exception StackTrace

```
java.lang.IllegalStateException: Duplicate key key1 (attempted merging values 0 and 1)
at java.base/java.util.stream.Collectors.duplicateKeyException(Collectors.java:135)
at java.base/java.util.stream.Collectors.lambda$uniqKeysMapAccumulator$0(Collectors.java:182)
at java.base/java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169)
at java.base/java.util.stream.IntPipeline$1$1.accept(IntPipeline.java:180)
at java.base/java.util.stream.Streams$RangeIntSpliterator.forEachRemaining(Streams.java:104)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:570)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:560)
at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:265)
at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:727)
at com.clickhouse.data.ClickHouseDataProcessor$DefaultSerDe.(ClickHouseDataProcessor.java:72)
at com.clickhouse.client.http.ApacheHttpConnectionImpl ...
at com.clickhouse.client.ClickHouseClient.executeAndWait(ClickHouseClient.java:894)
```

### Expected Behaviour

Duplicate column names are valid in ClickHouse/SQL, so building the response must not throw. Reading values **by position** should always work; name-based lookup of a duplicated name can reasonably return the first match (or be documented). Suggestion: build the name→index map with a merge function (keep the first index) instead of the throwing `Collectors.toMap`.

### Code Example

```java
import com.clickhouse.client.*;
import com.clickhouse.data.ClickHouseFormat;
import com.clickhouse.data.ClickHouseRecord;

public class DuplicateColumnRepro {
public static void main(String[] args) throws Exception {
ClickHouseNode server = ClickHouseNode.builder()
.host("localhost")
.port(ClickHouseProtocol.HTTP, 8123)
.build();

try (ClickHouseClient client = ClickHouseClient.newInstance(server.getProtocol());
ClickHouseResponse response = client.read(server)
.format(ClickHouseFormat.RowBinaryWithNamesAndTypes)
.query("SELECT key1, key1 FROM (SELECT 1 AS key1)") // duplicate output column name
.executeAndWait()) {
for (ClickHouseRecord r : response.records()) {
System.out.println(r.getValue(0).asString() + ", " + r.getValue(1).asString());
}
}
// throws: java.lang.IllegalStateException: Duplicate key key1 (attempted merging values 0 and 1)
}
}
```

**v2 — works (returns `1, 1`):**

### Configuration

#### Client Configuration
```java
client.read(server)
.format(ClickHouseFormat.RowBinaryWithNamesAndTypes)
// Reproduces with the default provider; also confirmed with both
// HttpConnectionProvider.APACHE_HTTP_CLIENT and HttpConnectionProvider.HTTP_URL_CONNECTION.
```

#### Environment
* [ ] Cloud
* Client version: 0.9.8 (`clickhouse-http-client` / `clickhouse-client` / `clickhouse-data`)
* Language version: Java 24.0.2 (also reproduces on Java 17)
* OS: macOS (Darwin arm64); also Linux

#### ClickHouse Server
* ClickHouse Server version: 26.3.10.60
* ClickHouse Server non-default settings, if any: none relevant
* `CREATE TABLE` statements for tables involved: none needed — `SELECT key1, key1 FROM (SELECT 1 AS key1)` reproduces it without any table
* Sample data for all these tables: n/a

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.