apache / apache/datafusion-sqlparser-rs
IDENTIFIER literal with double quotes inside single quotes is parsed as one token instead of multiple identifiers
- Dominant language
- Rust
- Stars
- 3.5k
- Forks
- 772
- Avg merge
- 4d 9h
- Merged PRs (30d)
- 17
Description
When running the following statement:
CREATE TABLE IDENTIFIER('"ORGDATA"."PUBLIC"."car_sales"')
the parser produces:
SingleQuotedString("\"ORGDATA\".\"PUBLIC\".\"car_sales\"")
It is treated as a single token (a literal string) instead of being parsed into three separate identifiers ("ORGDATA", "PUBLIC", "car_sales").
Since the IDENTIFIER function expects a literal, the content must be re-parsed.
**Suggestions:**
1. Expose a function that can take a SingleQuotedString like: '"ORGDATA"."PUBLIC"."car_sales"' and re-parse it into separate Ident tokens (ORGDATA, PUBLIC, car_sales).
2. change the tokenizer so that when IDENTIFIER is used with a quoted string containing double quotes, the inner content is tokenized as identifiers rather than treated as a raw single string.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by tracing how the parser handles CREATE TABLE IDENTIFIER(...) and how SingleQuotedString values reach the IDENTIFIER function or tokenizer. Reproduce the example and determine whether the intended result is three quoted identifiers; done means the input is no longer treated as one literal and existing parsing behavior remains intact.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- compilers
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100