[Bug] [PostgresSQL] pg-doris流式任务中断
- Dominant language
- Java
- Stars
- 9.7k
- Forks
- 2.4k
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 203
Description
### Search before asking
- [x] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues.
### What happened
我们从PostgreSQL到Doris的SeaTunnel流作业,每天凌晨0-1点左右都会因doris连接超时错误而失败。这项工作在一天的大部分时间里都能成功运行,但在清晨总是失败。我们doris库是新建的也不存在性能问题,报错的时间点也检查了doris环境以及各个be/fe节点均未发现异常。
### SeaTunnel Version
2.3.9
### SeaTunnel Config
```conf
"sink": [{
"fenodes": #{url},
"query-port": #{queryPort},
"username": #{user},
"password": #{password},
"database": #{database},
"table": "${table_name}",
"source_table_name" : #{source_table_name},
"sink.enable-2pc": "false",
"doris.batch.size": 10000,
"sink.buffer-size": 10485760,
"sink.check-interval": 50000,
"sink.max-retries": 5,
"sink.enable-delete":true,
"plugin_name" : "Doris",
"schema_save_mode":"ERROR_WHEN_SCHEMA_NOT_EXIST",
"doris.config": {
"format": "json",
"read_json_by_line": "true",
"num_as_string":"true"
}
}]
```
### Running Command
```shell
seatunnel.sh
```
### Error Exception
```log
java.lang.RuntimeException: java.lang.RuntimeException: java.util.concurrent.ExecutionException: org.apache.seatunnel.connectors.doris.exception.DorisConnectorException: ErrorCode:[Doris-01], ErrorDescription:[stream load error]
at org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.received(SinkFlowLifeCycle.java:302) ~[seatunnel-starter.jar:2.3.9]
at org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.received(SinkFlowLifeCycle.java:70) ~[seatunnel-starter.jar:2.3.9]
at org.apache.seatunnel.engine.server.task.SeaTunnelTransformCollector.collect(SeaTunnelTransformCollector.java:39) ~[seatunnel-starter.jar:2.3.9]
at org.apache.seatunnel.engine.server.task.SeaTunnelTransformCollector.collect(SeaTunnelTransformCollector.java:27) ~[seatunnel-starter.jar:2.3.9]
at org.apache.seatunnel.engine.server.task.group.queue.IntermediateBlockingQueue.handleRecord(IntermediateBlockingQueue.java:70) ~[seatunnel-starter.jar:2.3.9]
at org.apache.seatunnel.engine.server.task.group.queue.IntermediateBlockingQueue.collect(IntermediateBlockingQueue.java:50) ~[seatunnel-starter.jar:2.3.9]
at org.apache.seatunnel.engine.server.task.flow.IntermediateQueueFlowLifeCycle.collect(IntermediateQueueFlowLifeCycle.java:51) ~[seatunnel-starter.jar:2.3.9]
at org.apache.seatunnel.engine.server.task.TransformSeaTunnelTask.collect(TransformSeaTunnelTask.java:72) ~[seatunnel-starter.jar:2.3.9]
at org.apache.seatunnel.engine.server.task.SeaTunnelTask.stateProcess(SeaTunnelTask.java:169) ~[seatunnel-starter.jar:2.3.9]
at org.apache.seatunnel.engine.server.task.TransformSeaTunnelTask.call(TransformSeaTunnelTask.java:77) ~[seatunnel-starter.jar:2.3.9]
at org.apache.seatunnel.engine.server.TaskExecutionService$BlockingWorker.run(TaskExecutionService.java:694) ~[seatunnel-starter.jar:2.3.9]
at org.apache.seatunnel.engine.server.TaskExecutionService$NamedTaskWrapper.run(TaskExecutionService.java:1019) ~[seatunnel-starter.jar:2.3.9]
at org.apache.seatunnel.api.tracing.MDCRunnable.run(MDCRunnable.java:43) ~[seatunnel-starter.jar:2.3.9]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_451]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_451]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_451]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_451]
at java.lang.Thread.run(Thread.java:750) [?:1.8.0_451]
Caused by: java.lang.RuntimeException: java.util.concurrent.ExecutionException: org.apache.seatunnel.connectors.doris.exception.DorisConnectorException: ErrorCode:[Doris-01], ErrorDescription:[stream load error]
at org.apache.seatunnel.api.sink.multitablesink.MultiTableSinkWriter.prepareCommit(MultiTableSinkWriter.java:276) ~[seatunnel-starter.jar:2.3.9]
at org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.received(SinkFlowLifeCycle.java:201) ~[seatunnel-starter.jar:2.3.9]
... 17 more
Caused by: java.util.concurrent.ExecutionException: org.apache.seatunnel.connectors.doris.exception.DorisConnectorException: ErrorCode:[Doris-01], ErrorDescription:[stream load error]
at java.util.concurrent.FutureTask.report(FutureTask.java:122) ~[?:1.8.0_451]
at java.util.concurrent.FutureTask.get(FutureTask.java:192) ~[?:1.8.0_451]
at org.apache.seatunnel.api.sink.multitablesink.MultiTableSinkWriter.prepareCommit(MultiTableSinkWriter.java:274) ~[seatunnel-starter.jar:2.3.9]
at org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.received(SinkFlowLifeCycle.java:201) ~[seatunnel-starter.jar:2.3.9]
... 17 more
Caused by: org.apache.seatunnel.connectors.doris.exception.DorisConnectorException: ErrorCode:[Doris-01], ErrorDescription:[stream load error]
at org.apache.seatunnel.connectors.doris.sink.writer.DorisStreamLoad.stopLoad(DorisStreamLoad.java:228) ~[?:?]
at org.apache.seatunnel.connectors.doris.sink.writer.DorisSinkWriter.flush(DorisSinkWriter.java:221) ~[?:?]
at org.apache.seatunnel.connectors.doris.sink.writer.DorisSinkWriter.prepareCommit(DorisSinkWriter.java:206) ~[?:?]
at org.apache.seatunnel.api.sink.SinkWriter.prepareCommit(SinkWriter.java:75) ~[seatunnel-starter.jar:2.3.9]
at org.apache.seatunnel.api.sink.multitablesink.MultiTableSinkWriter.lambda$prepareCommit$4(MultiTableSinkWriter.java:259) ~[seatunnel-starter.jar:2.3.9]
... 6 more
Caused by: java.util.concurrent.ExecutionException: java.net.SocketException: 连接超时 (Read failed)
at java.util.concurrent.FutureTask.report(FutureTask.java:122) ~[?:1.8.0_451]
at java.util.concurrent.FutureTask.get(FutureTask.java:192) ~[?:1.8.0_451]
at org.apache.seatunnel.connectors.doris.sink.writer.DorisStreamLoad.stopLoad(DorisStreamLoad.java:226) ~[?:?]
at org.apache.seatunnel.connectors.doris.sink.writer.DorisSinkWriter.flush(DorisSinkWriter.java:221) ~[?:?]
at org.apache.seatunnel.connectors.doris.sink.writer.DorisSinkWriter.prepareCommit(DorisSinkWriter.java:206) ~[?:?]
at org.apache.seatunnel.api.sink.SinkWriter.prepareCommit(SinkWriter.java:75) ~[seatunnel-starter.jar:2.3.9]
at org.apache.seatunnel.api.sink.multitablesink.MultiTableSinkWriter.lambda$prepareCommit$4(MultiTableSinkWriter.java:259) ~[seatunnel-starter.jar:2.3.9]
... 6 more
```
### Zeta or Flink or Spark Version
_No response_
### Java or Scala Version
_No response_
### Screenshots
_No response_
### Are you willing to submit PR?
- [x] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with connectors/doris/src/main/java/org/apache/seatunnel/connectors/doris/sink/writer/DorisStreamLoad.java and DorisSinkWriter.java, then trace the failure through MultiTableSinkWriter.prepareCommit. Reproduce the scheduled PostgreSQL-to-Doris stream failure around midnight and collect the complete Doris error details. Done means the reported timeout is reproducible and its handling or cause is addressed with a regression test.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, postgresql
- Domain
- data-engineering, databases, stream-processing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100