ClickHouse / ClickHouse/clickhouse-java

java.sql.BatchUpdateException: Broken pipe (Write failed), server ClickHouseNode

Open
#1,620 9 comments 0 reactions 0 assignees View on GitHub
area:network bug
Dominant language
Java
Stars
1.6k
Forks
636
Avg merge
2d 16h
Merged PRs (30d)
28

Description

I write a Scala code using Spark to push data from a Hive table to ClickHouse. When I use the clickhouse-jdbc version 0.6.0, I set the following parameters.
```
properties.setProperty("http_connection_provider", HttpConnectionProvider.APACHE_HTTP_CLIENT.name());
properties.setProperty("socket_ip_tos", "32");
```
There is no problem when predicting 5kw data , but an exception is thrown when it becomes 5e data. threw an error as follows:
```
24/04/25 15:05:53 [task-result-getter-0] WARN TaskSetManager: Lost task 67.0 in stage 1.0 (TID 690, 11.11.10.76, executor 60): java.sql.BatchUpdateException: Broken pipe (Write failed), server ClickHouseNode [uri=http://sq02-ch-000119-clickhouse-17-1.local:8623/default, options={db=ge_order,socket_ip_tos=32,http_connection_provider=APACHE_HTTP_CLIENT}]@290467083
at com.clickhouse.jdbc.SqlExceptionUtils.batchUpdateError(SqlExceptionUtils.java:107)
at com.clickhouse.jdbc.internal.InputBasedPreparedStatement.executeAny(InputBasedPreparedStatement.java:154)
at com.clickhouse.jdbc.internal.AbstractPreparedStatement.executeLargeBatch(AbstractPreparedStatement.java:85)
at com.clickhouse.jdbc.internal.ClickHouseStatementImpl.executeBatch(ClickHouseStatementImpl.java:752)
at com.jd.clickhouse.spark.DataFrameExt$$anonfun$13$$anonfun$apply$23.apply(DataFrameExt.scala:513)
at com.jd.clickhouse.spark.DataFrameExt$$anonfun$13$$anonfun$apply$23.apply(DataFrameExt.scala:467)
```
When I adjust it to:
```
properties.setProperty("http_connection_provider", HttpConnectionProvider.HTTP_URL_CONNECTION.name())
```
threw an error as follows:
```
WARN TaskSetManager: Lost task 91.0 in stage 1.0 (TID 717, 10.198.62.132, executor 116): java.sql.BatchUpdateException: Error writing request body to server, server ClickHouseNode [uri=http://sq02-ch-000119-clickhouse-23-1.local:8623/default, options={db=ge_order,socket_ip_tos=32,http_connection_provider=HTTP_URL_CONNECTION}]@665046067
at com.clickhouse.jdbc.SqlExceptionUtils.batchUpdateError(SqlExceptionUtils.java:107)
at com.clickhouse.jdbc.internal.InputBasedPreparedStatement.executeAny(InputBasedPreparedStatement.java:154)
at com.clickhouse.jdbc.internal.AbstractPreparedStatement.executeLargeBatch(AbstractPreparedStatement.java:85)
at com.clickhouse.jdbc.internal.ClickHouseStatementImpl.executeBatch(ClickHouseStatementImpl.java:752)
at com.jd.clickhouse.spark.DataFrameExt$$anonfun$13$$anonfun$apply$23.apply(DataFrameExt.scala:513)
```

When I use the ru.yandex.clickhouse 0.1.34 ,The same problem will not occur.

The versions of ClickHouse, Spark version are all consistent.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.