Presto connector is performing full table scan on IS NULL predicate
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 195
Description
presto-connector is doing full table scan on `IS NULL` predicate.
Here is the query example:
`SELECT tool_version, tool_name, feature_type, operation_type_name
FROM map_labeling_logs
WHERE tool_name is null`
Here is the output of EXPLAIN query output:
`- Output[tool_version, tool_name, feature_type, operation_type_name] => [tool_version:varchar, tool_name:integer, feature_type:integer, operation_type_name:varchar]
Estimates: {rows: ? (?), cpu: ?, memory: 0.00, network: ?}
- RemoteStreamingExchange[GATHER] => [operation_type_name:varchar, tool_version:varchar, tool_name:integer, feature_type:integer]
Estimates: {rows: ? (?), cpu: ?, memory: 0.00, network: ?}
- ScanFilter[table = TableHandle {connectorId='pinot', connectorHandle='PinotTableHandle{connectorId=pinot, schemaName=default, tableName=map_labeling_logs, isQueryShort=Optional[false], expectedColumnHandles=Optional[[PinotColumnHandle{columnName=operation_type_name, dataType=varchar, type=REGULAR}, PinotColumnHandle{columnName=tool_version, dataType=varchar, type=REGULAR}, PinotColumnHandle{columnName=tool_name, dataType=integer, type=REGULAR}, PinotColumnHandle{columnName=feature_type, dataType=integer, type=REGULAR}]], pinotQuery=Optional[GeneratedPinotQuery{query=SELECT operation_type_name, tool_version, tool_name, feature_type FROM map_labeling_logs__TABLE_NAME_SUFFIX_TEMPLATE____TIME_BOUNDARY_FILTER_TEMPLATE__ LIMIT 2147483647, format=SQL, table=map_labeling_logs, expectedColumnIndices=[], groupByClauses=0, haveFilter=false, isQueryShort=false}]}', layout='Optional[PinotTableHandle{connectorId=pinot, schemaName=default, tableName=map_labeling_logs, isQueryShort=Optional[false], expectedColumnHandles=Optional[[PinotColumnHandle{columnName=operation_type_name, dataType=varchar, type=REGULAR}, PinotColumnHandle{columnName=tool_version, dataType=varchar, type=REGULAR}, PinotColumnHandle{columnName=tool_name, dataType=integer, type=REGULAR}, PinotColumnHandle{columnName=feature_type, dataType=integer, type=REGULAR}]], pinotQuery=Optional[GeneratedPinotQuery{query=SELECT operation_type_name, tool_version, tool_name, feature_type FROM map_labeling_logs__TABLE_NAME_SUFFIX_TEMPLATE____TIME_BOUNDARY_FILTER_TEMPLATE__ LIMIT 2147483647, format=SQL, table=map_labeling_logs, expectedColumnIndices=[], groupByClauses=0, haveFilter=false, isQueryShort=false}]}]'}, filterPredicate = IS_NULL(tool_name)] => [operation_type_name:varchar, tool_version:varchar, tool_name:integer, feature_type:integer]
Estimates: {rows: ? (?), cpu: ?, memory: 0.00, network: 0.00}/{rows: ? (?), cpu: ?, memory: 0.00, network: 0.00}
tool_version := PinotColumnHandle{columnName=tool_version, dataType=varchar, type=REGULAR}
tool_name := PinotColumnHandle{columnName=tool_name, dataType=integer, type=REGULAR}
operation_type_name := PinotColumnHandle{columnName=operation_type_name, dataType=varchar, type=REGULAR}
feature_type := PinotColumnHandle{columnName=feature_type, dataType=integer, type=REGULAR}`
Contributor guide
Research direction
Start by tracing the Presto connector's handling of the IS_NULL(tool_name) filter and the PinotTableHandle or GeneratedPinotQuery shown in EXPLAIN. Reproduce the query and verify that predicate pushdown produces a filtered Pinot query instead of a full table scan; completion should include regression coverage for this case.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, sql
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100