apache / apache/druid

"IN" and "NOT IN" with subquery in SQL queries

Open
#8,518 1 comment 0 reactions 0 assignees View on GitHub
Area - SQL
Dominant language
Java
Stars
14.1k
Forks
3.8k
Avg merge
2d 58m
Merged PRs (30d)
233

Description

### Affected Version

0.16.0 (and likely earlier too)

### Description

I noticed that the same issue was reported here: https://github.com/apache/incubator-druid/issues/8455, but was closed out. I am not sure why.

When I run the following query, it succeeds:

```sql
SELECT *
FROM myTable
WHERE
query IN(
SELECT
DISTINCT query
FROM
myTable
LIMIT 100
)
LIMIT 100
```

However, running the exact same query with NOT IN, rather than IN:
```sql
SELECT *
FROM myTable
WHERE
query NOT IN(
SELECT
DISTINCT query
FROM
myTable
LIMIT 100
)
LIMIT 100
```

I get the following error:

```
Unknown exception / Cannot build plan for query: SELECT * FROM (SELECT * FROM myTable WHERE query NOT IN( SELECT DISTINCT query FROM myTable LIMIT 100 ) LIMIT 100 ) LIMIT 100 / org.apache.druid.java.util.common.ISE
```

I have not been able to find any other error in the logs for the Broker. The error above shows up in the query UI.

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the reported IN and NOT IN queries against Apache Druid 0.16.0, using the subquery and LIMIT clauses shown in the issue. Compare the query-planning behavior and error output; done means the NOT IN form plans and executes without the reported Cannot build plan exception.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, sql
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.