apache / apache/pinot

PrestoDB/Trino connector cannot connect to Pinot after Pinot cluster updates its instance hostname

Open
#7,837 1 comment 1 reaction 0 assignees View on GitHub
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
1d 21h
Merged PRs (30d)
189

Description

If the Pinot cluster adopts https://github.com/apache/pinot/pull/7064 to update its Helix hostname into to Pinot Helix InstanceConfig, the PrestoDB/Trino connector will not be able to connect to Pinot any more.

The main reason is inside PrestoDB/Trino code, that it tries to derive `hostname:port` connection string to each Pinot server instance by using the traditional naming convention.

For example, traditionally, if our server instance name in Pinot is `Server_fw-1.service.consul_8099`, PrestoDB/Trino directly regex-parse the name to find out that it should connect to Pinot Server via `fw-1.service.consul:8099`. This works fine if the Pinot Server registers itself under DNS with FQDN name as `fw-1.service.consul:8099`; however, if we adopt https://github.com/apache/pinot/pull/7064 to update Helix instance `Server_fw-1.service.consul_8099` to point to `"hostname":"pinotserver--a073de1ef0.awsinstance.io", "port":"8099"`, PrestoDB/Trino will **NOT** query Helix instance to get that FQDN `pinotserver--a073de1ef0.awsinstance.io:8099`, let alone connects to the updated hostname:port.

We need to update PrestoDB/Trino connector code to properly query Pinot controller to resolve the host name and then later connect to the resolved host.

A bit extra note after my failed attempt on a "quick fix" in Trino connector:
Trino queries routing table for a table, and then for each segment, it gets ServerName for that segment, and then RegEx to parse out the hostname:port to connect directly.

cc @xiangfu0 @jadami10

Contributor guide

Open the contributing guide

Research direction

Start in the Trino connector path that reads the routing table, obtains each segment's ServerName, and regex-parses a host and port. Trace how the Pinot controller exposes the instance hostname and compare that with the current direct parsing flow. Done means the connector uses the resolved hostname and port when querying each Pinot server, including hostnames that differ from the Helix instance name.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.