ClickHouse / ClickHouse/clickhouse-java
Enhance insert() API to support SELECT with input() for computed columns v2
- Lenguaje dominante
- Java
- Estrellas
- 1.6k
- Forks
- 636
- Merge medio
- 2 d 23 h
- PR fusionados (30 d)
- 29
Descripción
### Current Limitation
The `insert()` method only supports direct table insertion, not SELECT statements
with transformations.
### Desired Enhancement
Allow `insert()` to accept SELECT statements with input() function:
```java
client.insert()
.table("target_table")
.query("SELECT col1, col2, col1 + col2 as computed FROM input('col1 Int32, col2 Int32')")
.data(inputStream)
.format(ClickHouseFormat.CSV)
.send();
```
Or alternative syntax:
```java
client.insert("INSERT INTO target_table SELECT col1, col2, col1 + col2 as computed FROM input('col1 Int32, col2 Int32')")
.data(inputStream)
.format(ClickHouseFormat.CSV)
.send();
```
### Use Case
We need to stream data with computed columns during insertion:
- Stream large datasets efficiently
- Apply transformations/computed columns on the fly
- Leverage ClickHouse's input() function capabilities
- Maintain the performance benefits of the streaming insert() API
### Business Value
This is a common pattern in ClickHouse for ETL workloads where data needs
transformation during insertion without materializing intermediate results.
### Current Workaround (Working but Undocumented)
We're successfully using this pattern in production:
```java
String loadQuery = tableName + "(" + headers + ")" +
" select " + parametersQuery + "* from input('" + csvHeadersStructure + "')";
try (InputStream inputStream = Files.newInputStream(Paths.get(filepath))) {
InsertResponse response = client.insert(loadQuery, inputStream, ClickHouseFormat.CSVWithNames)
.get(1200, TimeUnit.SECONDS);
}
```
**This works but:**
- Not documented in official API docs
- Unclear if this is intended behavior or might break in future versions
- No IDE autocomplete/type safety for this pattern
Guía de contribución
Línea de trabajo
Comienza por los puntos de entrada de la API insert() del cliente Java y el manejo existente de las cadenas de consulta; después, compara las formas builder y overload con el workaround funcional de input() descrito aquí. Define y prueba la sintaxis de SELECT/input() compatible con datos en streaming y columnas calculadas, y documenta si el patrón existente es intencionado.
Escrito por el modelo de indexación a partir del texto del issue.
Evaluación
- Stack tecnológico
- java
- Área
- api, backend-api-design
- Tipo de issue
- Nueva funcionalidad
- Dificultad
- 4/5
- Tiempo estimado
- 3-5 días
- Estado de actividad
- Tranquilo
- Claridad
- Bastante claro
- Aptitud para principiantes
- 48/100