ClickHouse / ClickHouse/clickhouse-java
java.sql.BatchUpdateException: Broken pipe (Write failed), server ClickHouseNode
- Dominant language
- Java
- Stars
- 1.6k
- Forks
- 636
- Avg merge
- 2d 16h
- Merged PRs (30d)
- 28
Description
I write a Scala code using Spark to push data from a Hive table to ClickHouse. When I use the clickhouse-jdbc version 0.6.0, I set the following parameters.
```
properties.setProperty("http_connection_provider", HttpConnectionProvider.APACHE_HTTP_CLIENT.name());
properties.setProperty("socket_ip_tos", "32");
```
There is no problem when predicting 5kw data , but an exception is thrown when it becomes 5e data. threw an error as follows:
```
24/04/25 15:05:53 [task-result-getter-0] WARN TaskSetManager: Lost task 67.0 in stage 1.0 (TID 690, 11.11.10.76, executor 60): java.sql.BatchUpdateException: Broken pipe (Write failed), server ClickHouseNode [uri=http://sq02-ch-000119-clickhouse-17-1.local:8623/default, options={db=ge_order,socket_ip_tos=32,http_connection_provider=APACHE_HTTP_CLIENT}]@290467083
at com.clickhouse.jdbc.SqlExceptionUtils.batchUpdateError(SqlExceptionUtils.java:107)
at com.clickhouse.jdbc.internal.InputBasedPreparedStatement.executeAny(InputBasedPreparedStatement.java:154)
at com.clickhouse.jdbc.internal.AbstractPreparedStatement.executeLargeBatch(AbstractPreparedStatement.java:85)
at com.clickhouse.jdbc.internal.ClickHouseStatementImpl.executeBatch(ClickHouseStatementImpl.java:752)
at com.jd.clickhouse.spark.DataFrameExt$$anonfun$13$$anonfun$apply$23.apply(DataFrameExt.scala:513)
at com.jd.clickhouse.spark.DataFrameExt$$anonfun$13$$anonfun$apply$23.apply(DataFrameExt.scala:467)
```
When I adjust it to:
```
properties.setProperty("http_connection_provider", HttpConnectionProvider.HTTP_URL_CONNECTION.name())
```
threw an error as follows:
```
WARN TaskSetManager: Lost task 91.0 in stage 1.0 (TID 717, 10.198.62.132, executor 116): java.sql.BatchUpdateException: Error writing request body to server, server ClickHouseNode [uri=http://sq02-ch-000119-clickhouse-23-1.local:8623/default, options={db=ge_order,socket_ip_tos=32,http_connection_provider=HTTP_URL_CONNECTION}]@665046067
at com.clickhouse.jdbc.SqlExceptionUtils.batchUpdateError(SqlExceptionUtils.java:107)
at com.clickhouse.jdbc.internal.InputBasedPreparedStatement.executeAny(InputBasedPreparedStatement.java:154)
at com.clickhouse.jdbc.internal.AbstractPreparedStatement.executeLargeBatch(AbstractPreparedStatement.java:85)
at com.clickhouse.jdbc.internal.ClickHouseStatementImpl.executeBatch(ClickHouseStatementImpl.java:752)
at com.jd.clickhouse.spark.DataFrameExt$$anonfun$13$$anonfun$apply$23.apply(DataFrameExt.scala:513)
```
When I use the ru.yandex.clickhouse 0.1.34 ,The same problem will not occur.
The versions of ClickHouse, Spark version are all consistent.
Contributor guide
Assessment
This issue has not been assessed yet.