apache / apache/datafusion-sqlparser-rs

Support for `IN $placeholder` syntax

Open
#1,962 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
3.5k
Forks
772
Avg merge
4d 9h
Merged PRs (30d)
17

Description

Some DBs support using placeholders for `IN` clauses in prepared statements. For example in DuckDB:

```
$ duckdb
DuckDB v1.3.0 (Ossivalis) 71c5c07cdd
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database.
D PREPARE qry AS SELECT 'a' in $list;
D EXECUTE qry(list := ['a', 'b']);
┌──────────────────────┐
│ contains($list, 'a') │
│ boolean │
├──────────────────────┤
│ true │
└──────────────────────┘
```

However, currently the parser doesn't handle this, for example:

```rust
#[test]
fn test_parse_in_placeholder() {
let stmt = all_dialects().verified_stmt("SELECT i IN $placeholder");
dbg!(&stmt);
}
```

Fails w/ `SELECT i IN $placeholder: ParserError("Expected: (, found: $placeholder")`.

To fix this, I think we would need to make two changes:

First off, the definition of `Expr::InList`:

```rust
/// `[ NOT ] IN (val1, val2, ...)`
InList {
expr: Box,
list: Vec,
negated: bool,
},
```

The issue is that `list` is always a Vec, where in this case we want list to be a `Expr::Value` w/ `value = Placeholder("$placeholder")`.

If `InList` supports that, then in `parse_in` we can do a check like this before the `expect_token(LParen)`:

```rust
if let Token::Placeholder(_) = &self.peek_token_ref().token {
let placeholder = self.parse_expr()?;
return Ok(Expr::InList {
expr: Box::new(expr),
list: placeholder,
negated,
})
};
self.expect_token(&Token::LParen)?;
```

But, how can we cleanly support this, without too much breakage for existing consumers? Note that I considered just putting the placeholder inside the list, but that doesn't work since that would represent `IN ($placeholder)` which has a very different meaning.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the Expr::InList definition and the parser's parse_in logic, then run the provided all_dialects().verified_stmt("SELECT i IN $placeholder") case to reproduce the failure. Determine a representation that distinguishes IN $placeholder from IN ($placeholder) while limiting breakage for existing consumers, and add coverage showing both forms parse correctly.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust, sql
Domain
compilers, databases
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.