apache / apache/doris-flink-connector
[Bug] stream load 报 HTTP/1.1 400 No Host
- Dominant language
- Java
- Stars
- 385
- Forks
- 279
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 6
Description
### Search before asking
- [x] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues.
### Version
flink-doris-connector-1.17-24.1.0
### What's Wrong?
org.apache.doris.flink.sink.batch.DorisBatchStreamLoad - trigger flush by buffer full, flush: true
2025-03-05 18:01:40.086 [pool-1-streamload-executor-1] INFO org.apache.doris.flink.sink.batch.DorisBatchStreamLoad - stream load started for cdp_label_flink_cdc_1741167889103_0_lyh_test_c_20250305_target_6f4bc24c-3fbc-410a-b224-fa574eca3281 on host xxx:8030
2025-03-05 18:01:40.127 [pool-1-streamload-executor-1] ERROR org.apache.doris.flink.sink.batch.DorisBatchStreamLoad - stream load failed with xxx:8030, reason HTTP/1.1 400 No Host, to retry
2025-03-05 18:01:40.134 [pool-1-streamload-executor-1] INFO org.apache.doris.flink.sink.batch.DorisBatchStreamLoad - stream load started for cdp_label_flink_cdc_1741167889103_0_lyh_test_c_20250305_target_6f4bc24c-3fbc-410a-b224-fa574eca3281_1 on host xxx:8030
2025-03-05 18:01:40.140 [pool-1-streamload-executor-1] ERROR org.apache.doris.flink.sink.batch.DorisBatchStreamLoad - stream load failed with xxx:8030, reason HTTP/1.1 400 No Host, to retry
2025-03-05 18:01:40.146 [pool-1-streamload-executor-1] INFO org.apache.doris.flink.sink.batch.DorisBatchStreamLoad - stream load started for cdp_label_flink_cdc_1741167889103_0_lyh_test_c_20250305_target_6f4bc24c-3fbc-410a-b224-fa574eca3281_2 on host xxx:8030
2025-03-05 18:01:40.151 [pool-1-streamload-executor-1] ERROR org.apache.doris.flink.sink.batch.DorisBatchStreamLoad - stream load failed with xxx:8030, reason HTTP/1.1 400 No Host, to retry
2025-03-05 18:01:40.158 [pool-1-streamload-executor-1] INFO org.apache.doris.flink.sink.batch.DorisBatchStreamLoad - stream load started for cdp_label_flink_cdc_1741167889103_0_lyh_test_c_20250305_target_6f4bc24c-3fbc-410a-b224-fa574eca3281_3 on host xxx:8030
2025-03-05 18:01:40.163 [pool-1-streamload-executor-1] ERROR org.apache.doris.flink.sink.batch.DorisBatchStreamLoad - stream load failed with xxx:8030, reason HTTP/1.1 400 No Host, to retry
2025-03-05 18:01:40.212 [pool-1-streamload-executor-1] ERROR org.apache.doris.flink.sink.batch.DorisBatchStreamLoad - worker running error
org.apache.doris.flink.exception.DorisBatchLoadException: stream load error: stream load failed with: HTTP/1.1 400 No Host
at org.apache.doris.flink.sink.batch.DorisBatchStreamLoad$LoadAsyncExecutor.load(DorisBatchStreamLoad.java:527)
at org.apache.doris.flink.sink.batch.DorisBatchStreamLoad$LoadAsyncExecutor.run(DorisBatchStreamLoad.java:413)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: org.apache.doris.flink.exception.DorisRuntimeException: stream load failed with: HTTP/1.1 400 No Host
at org.apache.doris.flink.sink.batch.DorisBatchStreamLoad$LoadAsyncExecutor.load(DorisBatchStreamLoad.java:509)
... 4 common frames omitted
2025-03-05 18:01:40.212 [pool-1-streamload-executor-1] INFO org.apache.doris.flink.sink.batch.DorisBatchStreamLoad - LoadAsyncExecutor stop
2025-03-05 18:01:40.215 [Data Collapse Processor -> Map -> Doris Sink By StreamLoad: Writer -> Doris Sink By StreamLoad: Committer (1/1)#0] INFO org.apache.doris.flink.sink.batch.DorisBatchWriter - DorisBatchWriter Close
2025-03-05 18:01:40.368 [Data Collapse Processor -> Map -> Doris Sink By StreamLoad: Writer -> Doris Sink By StreamLoad: Committer (1/1)#0] INFO o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Closed RocksDB State Backend. Cleaning up RocksDB working directory /alidata1/tmp/nm/usercache/bigdata/appcache/application_1741157851668_0168/tm_container_e1863_1741157851668_0168_01_000002/tmp/job_8dc5a750c994bbb8e71349a5d99c2a16_op_KeyedProcessOperator_e0d9683bb400b8c0fc363829cca7469b__1_1__uuid_a6938353-0b3f-4a54-821f-3dd14579bafe.
2025-03-05 18:01:40.373 [Data Collapse Processor -> Map -> Doris Sink By StreamLoad: Writer -> Doris Sink By StreamLoad: Committer (1/1)#0] WARN org.apache.flink.runtime.taskmanager.Task - Data Collapse Processor -> Map -> Doris Sink By StreamLoad: Writer -> Doris Sink By StreamLoad: Committer (1/1)#0 (d4a96ed981a8d1c855102c78cccea54e_e0d9683bb400b8c0fc363829cca7469b_0_0) switched from RUNNING to FAILED with failure cause:
org.apache.doris.flink.exception.DorisBatchLoadException: org.apache.doris.flink.exception.DorisBatchLoadException: stream load error: stream load failed with: HTTP/1.1 400 No Host
at org.apache.doris.flink.sink.batch.DorisBatchStreamLoad.checkFlushException(DorisBatchStreamLoad.java:307)
at org.apache.doris.flink.sink.batch.DorisBatchStreamLoad.writeRecord(DorisBatchStreamLoad.java:182)
at org.apache.doris.flink.sink.batch.DorisBatchWriter.writeOneDorisRecord(DorisBatchWriter.java:154)
at org.apache.doris.flink.sink.batch.DorisBatchWriter.write(DorisBatchWriter.java:120)
at org.apache.flink.streaming.runtime.operators.sink.SinkWriterOperator.processElement(SinkWriterOperator.java:158)
at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.pushToOperator(CopyingChainingOutput.java:75)
at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:50)
at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:29)
at org.apache.flink.streaming.api.operators.StreamMap.processElement(StreamMap.java:38)
at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.pushToOperator(CopyingChainingOutput.java:75)
at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:50)
at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:29)
at org.apache.flink.streaming.api.operators.TimestampedCollector.collect(TimestampedCollector.java:51)
at org.apache.flink.streaming.api.operators.KeyedProcessOperator.processElement(KeyedProcessOperator.java:83)
at org.apache.flink.streaming.runtime.io.RecordProcessorUtils.lambda$getRecordProcessor$0(RecordProcessorUtils.java:60)
at org.apache.flink.streaming.runtime.tasks.OneInputStreamTask$StreamTaskNetworkOutput.emitRecord(OneInputStreamTask.java:237)
at org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.processElement(AbstractStreamTaskNetworkInput.java:146)
at org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.emitNext(AbstractStreamTaskNetworkInput.java:110)
at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65)
at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:550)
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:231)
at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:839)
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:788)
at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:952)
at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:931)
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:745)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:562)
at java.lang.Thread.run(Thread.java:750)
Caused by: org.apache.doris.flink.exception.DorisBatchLoadException: stream load error: stream load failed with: HTTP/1.1 400 No Host
at org.apache.doris.flink.sink.batch.DorisBatchStreamLoad$LoadAsyncExecutor.load(DorisBatchStreamLoad.java:527)
at org.apache.doris.flink.sink.batch.DorisBatchStreamLoad$LoadAsyncExecutor.run(DorisBatchStreamLoad.java:413)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
... 1 common frames omitted
Caused by: org.apache.doris.flink.exception.DorisRuntimeException: stream load failed with: HTTP/1.1 400 No Host
at org.apache.doris.flink.sink.batch.DorisBatchStreamLoad$LoadAsyncExecutor.load(DorisBatchStreamLoad.java:509)
... 4 common frames omitted
### What You Expected?
重试时没有间隔时间参数,看不到statusCode,这个是什么问题导致的,小概率事件,且出现后会阻塞正常的后续写doris流程
### How to Reproduce?
_No response_
### Anything Else?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
Contributor guide
Research direction
Start with org.apache.doris.flink.sink.batch.DorisBatchStreamLoad, especially the failure paths around lines 182, 307, 413, 509, and 527 shown in the stack trace. Trace how the stream-load request and retry are handled, then inspect existing connector tests for error handling. Done means the HTTP 400 case is understood and no longer blocks subsequent Doris writes, with coverage for the observed failure behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering, stream-processing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100