[Improve][Connector-V2][Starrocks-Source]When obtaining data from Starrock, an exception prompt 'Unknown HostException' appears
- Dominant language
- Java
- Stars
- 9.7k
- Forks
- 2.4k
- Avg merge
- 3d 9h
- Merged PRs (30d)
- 204
Description
When I retrieve data from Starrocks, the configuration is as follows:
```
env {
parallelism = 1
job.mode = "BATCH"
spark.app.name = "SeaTunnel"
spark.executor.instances = 2
spark.executor.cores = 1
spark.executor.memory = "1g"
spark.master = local
}
source {
StarRocks {
nodeUrls = ["localhost:32529"]
username = xxxx
password = "xxxx"
database = "db01"
table = "user"
max_retries = 3
schema {
fields {
name1 = STRING
name2 = STRING
name3 = STRING
}
}
}
}
transform {
}
sink {
Console {
parallelism = 1
}
}
```
At this point, an error log will appear:
```
Caused by: org.apache.seatunnel.connectors.seatunnel.starrocks.exception.StarRocksConnectorException: ErrorCode:[STARROCKS-04], ErrorDescription:[Create StarRocks BE reader failed] - Failed to open socket
at org.apache.seatunnel.connectors.seatunnel.starrocks.client.source.StarRocksBeReadClient.(StarRocksBeReadClient.java:88)
at org.apache.seatunnel.connectors.seatunnel.starrocks.source.StarRocksSourceReader.read(StarRocksSourceReader.java:113)
at org.apache.seatunnel.connectors.seatunnel.starrocks.source.StarRocksSourceReader.pollNext(StarRocksSourceReader.java:75)
at org.apache.seatunnel.translation.source.ParallelSource.run(ParallelSource.java:144)
at org.apache.seatunnel.translation.spark.source.partition.batch.ParallelBatchPartitionReader.lambda$prepare$0(ParallelBatchPartitionReader.java:117)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
at java.util.concurrent.FutureTask.run(FutureTask.java)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
... 3 more
Caused by: com.starrocks.shade.org.apache.thrift.transport.TTransportException: java.net.UnknownHostException: pingt-7f5cf4cfdc-cn-0.pingt-7f5cf4cfdc-cn-headless.olap-c9cd-11f0-aec5-c245473c61c7-61f3bdda.svc.cluster.local
at com.starrocks.shade.org.apache.thrift.transport.TSocket.open(TSocket.java:226)
at org.apache.seatunnel.connectors.seatunnel.starrocks.client.source.StarRocksBeReadClient.(StarRocksBeReadClient.java:85)
... 12 more
Caused by: java.net.UnknownHostException: pingt-7f5cf4cfdc-cn-0.pingt-7f5cf4cfdc-cn-headless.olap-c9cd-11f0-aec5-c245473c61c7-61f3bdda.svc.cluster.local
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:184)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at com.starrocks.shade.org.apache.thrift.transport.TSocket.open(TSocket.java:221)
... 13 more
```
I noticed that the code obtains the host and port 9060 of BE through the FE address, and returns the domain name. However, our starrocks are deployed through k8s and are not in the same cluster as our Spark cluster, which can cause an exception:
My solution idea is that the host and port 9060 of BE can be obtained through configuration, or if the user has not configured this information, it can be obtained through FE information. What do you think?
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.