apache / apache/druid

Remove Bindable query path

Open
#9,896 1 comment 0 reactions 0 assignees View on GitHub
Area - SQL
Dominant language
Java
Stars
14.1k
Forks
3.8k
Avg merge
2d 58m
Merged PRs (30d)
233

Description

Remove the Bindable query path, meaning all SQL queries can be executed as native queries (or a sequence of concatenated native queries, in the case of UNION ALL). Currently, this path is used for the `sys` and `INFORMATION_SCHEMA` schemas. It is also used when the planner swaps in static `Values` for a table scan, which happens when it realizes that a `WHERE` filter can never possibly match anything.

This would allow supporting special Druid functions, like `TIME_PARSE`, on these tables. IMO this is the main benefit. There are also side benefits: like simplifying the SQL planning process, and joining between system tables and regular tables.

The approach that comes to mind first is to have the SQL layer generate lazy inline datasources for these special tables using `InlineDataSource.fromIterable`. We need to make sure we don't blow through memory or other resources limits while doing this. (The current Bindable stuff has no resource-limiting facility, so this would be an improvement.)

I believe the main prerequisite here is supporting `ORDER BY` on non-aggregating queries, meaning adding sort-by-non-time to the Scan query. It's possible that we could start this off by only supporting it for cases where there is just a single segment involved in the query. I believe this would cover the `sys` and `INFORMATION_SCHEMA` cases.

Contributor guide

Open the contributing guide

Research direction

Start by tracing the SQL layer's Bindable query path and the Scan query prerequisite described in the issue, including system and INFORMATION_SCHEMA handling and static Values substitution. Done means all SQL queries use native queries or concatenated native queries for UNION ALL, support the required non-aggregating ORDER BY behavior, and enforce resource limits for inline datasources.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, sql
Domain
backend-api-design, databases
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.