starrocks-writer在连接Fe节点时,能不能修改默认超时时间为30S,否则单个Fe节点的集群在写数据时容易报:connect time out
- Dominant language
- Java
- Stars
- 17.4k
- Forks
- 5.7k
- PR merge metrics
- No merged PRs in 30d
Description
当使用starrocks-writer插件写数据至sr集群的时候,完全没有规律的会报一下异常
: Failed to connect to address:http://x.x.x.x:8030 java.net.SocketTimeOutException: connect time out
会导致整个datax的采集任务失败;
当我查看了下starrocks-writer插件源代码,发现每次写数据至sr集群时,会测试当前fe节点是否正常连接,我把这段代码复制出来进行单独测试,发现平均1K次会有0-5次左右失败,错误和上面的异常是能对应的;最终参考flink-starrocks插件,代码都相同,只是超时时间datax设置的1s,flink设置的是30秒,当把datax的超时时间修改为30s后,生产环境就比较稳定。
在sr集群中fe节点只有1个的时候这种情况比较常见。如果是3个节点那大概率是不会发生的。
所以能不能将datax中设置超时时间和flink中保持一致,都默认设置为30S?
Contributor guide
No contributing guide indexed for this repository
Research direction
Start in the starrocks-writer plugin source where the FE node connection is checked, and compare its timeout with the referenced flink-starrocks implementation. Set the default connection timeout to 30 seconds and verify that the writer handles single-FE connection attempts without the reported timeout failures.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100