ClickHouse / ClickHouse/clickhouse-java
client-v1 (clickhouse-data): "Duplicate key" IllegalStateException on result sets with duplicate column names (0.9.8)
- Langage dominant
- Java
- Étoiles
- 1.6k
- Forks
- 636
- Merge moyen
- 2 j 23 h
- PR mergées (30 j)
- 29
Description
## Description
The v1 client (`clickhouse-client` / `clickhouse-http-client` **0.9.8**) throws `java.lang.IllegalStateException: Duplicate key ...` when a query returns a result set with **duplicate column names** (which is valid SQL).
`com.clickhouse.data.ClickHouseDataProcessor$DefaultSerDe` builds a *column-name → index* map with `Collectors.toMap` (no merge function) at construction time, so any repeated column name fails immediately — **before the caller can read anything, even when reading purely by position**.
This reproduces with `RowBinaryWithNamesAndTypes` (the format that carries column names). A trivial `SELECT key1, key1 FROM (SELECT 1 AS key1)` is enough. Real-world trigger: a reporting tool that lets a user select the same projection twice (e.g. two `Date` columns), producing SQL like `SELECT toDate(...) AS Date, toDate(...) AS Date, ...`.
This looks related to #2336 (reported on 0.8.4, marked fixed in 0.9.3 via #2618), but that fix was in the JDBC `TableSchema` (Guava `ImmutableMap`, *"Multiple entries with same key"*). The failure here is a **different code path** — `ClickHouseDataProcessor` using the JDK `Collectors.toMap` (*"attempted merging values"*) — and **still fails in 0.9.8**.
### Steps to reproduce
1. Use `com.clickhouse:clickhouse-http-client:0.9.8` (v1 client).
2. Execute a query whose result has two columns with the same name, e.g. `SELECT key1, key1 FROM (SELECT 1 AS key1)`, with format `RowBinaryWithNamesAndTypes`.
3. `executeAndWait()` / iterating `response.records()` throws.
### Error Log or Exception StackTrace
```
java.lang.IllegalStateException: Duplicate key key1 (attempted merging values 0 and 1)
at java.base/java.util.stream.Collectors.duplicateKeyException(Collectors.java:135)
at java.base/java.util.stream.Collectors.lambda$uniqKeysMapAccumulator$0(Collectors.java:182)
at java.base/java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169)
at java.base/java.util.stream.IntPipeline$1$1.accept(IntPipeline.java:180)
at java.base/java.util.stream.Streams$RangeIntSpliterator.forEachRemaining(Streams.java:104)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:570)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:560)
at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:265)
at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:727)
at com.clickhouse.data.ClickHouseDataProcessor$DefaultSerDe.(ClickHouseDataProcessor.java:72)
at com.clickhouse.client.http.ApacheHttpConnectionImpl ...
at com.clickhouse.client.ClickHouseClient.executeAndWait(ClickHouseClient.java:894)
```
### Expected Behaviour
Duplicate column names are valid in ClickHouse/SQL, so building the response must not throw. Reading values **by position** should always work; name-based lookup of a duplicated name can reasonably return the first match (or be documented). Suggestion: build the name→index map with a merge function (keep the first index) instead of the throwing `Collectors.toMap`.
### Code Example
```java
import com.clickhouse.client.*;
import com.clickhouse.data.ClickHouseFormat;
import com.clickhouse.data.ClickHouseRecord;
public class DuplicateColumnRepro {
public static void main(String[] args) throws Exception {
ClickHouseNode server = ClickHouseNode.builder()
.host("localhost")
.port(ClickHouseProtocol.HTTP, 8123)
.build();
try (ClickHouseClient client = ClickHouseClient.newInstance(server.getProtocol());
ClickHouseResponse response = client.read(server)
.format(ClickHouseFormat.RowBinaryWithNamesAndTypes)
.query("SELECT key1, key1 FROM (SELECT 1 AS key1)") // duplicate output column name
.executeAndWait()) {
for (ClickHouseRecord r : response.records()) {
System.out.println(r.getValue(0).asString() + ", " + r.getValue(1).asString());
}
}
// throws: java.lang.IllegalStateException: Duplicate key key1 (attempted merging values 0 and 1)
}
}
```
**v2 — works (returns `1, 1`):**
### Configuration
#### Client Configuration
```java
client.read(server)
.format(ClickHouseFormat.RowBinaryWithNamesAndTypes)
// Reproduces with the default provider; also confirmed with both
// HttpConnectionProvider.APACHE_HTTP_CLIENT and HttpConnectionProvider.HTTP_URL_CONNECTION.
```
#### Environment
* [ ] Cloud
* Client version: 0.9.8 (`clickhouse-http-client` / `clickhouse-client` / `clickhouse-data`)
* Language version: Java 24.0.2 (also reproduces on Java 17)
* OS: macOS (Darwin arm64); also Linux
#### ClickHouse Server
* ClickHouse Server version: 26.3.10.60
* ClickHouse Server non-default settings, if any: none relevant
* `CREATE TABLE` statements for tables involved: none needed — `SELECT key1, key1 FROM (SELECT 1 AS key1)` reproduces it without any table
* Sample data for all these tables: n/a
Guide de contribution
Ouvrir le guide de contribution
Piste de recherche
Commencez dans ClickHouseDataProcessor$DefaultSerDe, à l’emplacement du constructeur indiqué dans la stack trace, et reproduisez le problème avec RowBinaryWithNamesAndTypes en utilisant la requête fournie comportant des colonnes dupliquées. Vérifiez comment la table de correspondance nom de colonne-index gère les noms répétés et vérifiez que les lectures positionnelles s’effectuent jusqu’au bout avec succès. C’est terminé lorsque les noms de sortie dupliqués n’empêchent plus la construction de la réponse ni l’accès basé sur la position.
Rédigé par le modèle d'indexation à partir du texte de l'issue.
Évaluation
- Stack technique
- java
- Domaine
- api
- Type d'issue
- Bug
- Difficulté
- 2/5
- Temps estimé
- 1-3 heures
- Activité
- Calme
- Clarté
- Clairement spécifiée
- Accessibilité débutants
- 72/100