PrestoDB/Trino connector cannot connect to Pinot after Pinot cluster updates its instance hostname
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 189
Description
If the Pinot cluster adopts https://github.com/apache/pinot/pull/7064 to update its Helix hostname into to Pinot Helix InstanceConfig, the PrestoDB/Trino connector will not be able to connect to Pinot any more.
The main reason is inside PrestoDB/Trino code, that it tries to derive `hostname:port` connection string to each Pinot server instance by using the traditional naming convention.
For example, traditionally, if our server instance name in Pinot is `Server_fw-1.service.consul_8099`, PrestoDB/Trino directly regex-parse the name to find out that it should connect to Pinot Server via `fw-1.service.consul:8099`. This works fine if the Pinot Server registers itself under DNS with FQDN name as `fw-1.service.consul:8099`; however, if we adopt https://github.com/apache/pinot/pull/7064 to update Helix instance `Server_fw-1.service.consul_8099` to point to `"hostname":"pinotserver--a073de1ef0.awsinstance.io", "port":"8099"`, PrestoDB/Trino will **NOT** query Helix instance to get that FQDN `pinotserver--a073de1ef0.awsinstance.io:8099`, let alone connects to the updated hostname:port.
We need to update PrestoDB/Trino connector code to properly query Pinot controller to resolve the host name and then later connect to the resolved host.
A bit extra note after my failed attempt on a "quick fix" in Trino connector:
Trino queries routing table for a table, and then for each segment, it gets ServerName for that segment, and then RegEx to parse out the hostname:port to connect directly.
cc @xiangfu0 @jadami10
Contributor guide
Research direction
Start in the Trino connector path that reads the routing table, obtains each segment's ServerName, and regex-parses a host and port. Trace how the Pinot controller exposes the instance hostname and compare that with the current direct parsing flow. Done means the connector uses the resolved hostname and port when querying each Pinot server, including hostnames that differ from the Helix instance name.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100