apache / apache/datafusion-sqlparser-rs

IDENTIFIER literal with double quotes inside single quotes is parsed as one token instead of multiple identifiers

Open
#2,048 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
3.5k
Forks
772
Avg merge
4d 9h
Merged PRs (30d)
17

Description

When running the following statement:

CREATE TABLE IDENTIFIER('"ORGDATA"."PUBLIC"."car_sales"')

the parser produces:

SingleQuotedString("\"ORGDATA\".\"PUBLIC\".\"car_sales\"")

It is treated as a single token (a literal string) instead of being parsed into three separate identifiers ("ORGDATA", "PUBLIC", "car_sales").
Since the IDENTIFIER function expects a literal, the content must be re-parsed.

**Suggestions:**

1. Expose a function that can take a SingleQuotedString like: '"ORGDATA"."PUBLIC"."car_sales"' and re-parse it into separate Ident tokens (ORGDATA, PUBLIC, car_sales).

2. change the tokenizer so that when IDENTIFIER is used with a quoted string containing double quotes, the inner content is tokenized as identifiers rather than treated as a raw single string.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by tracing how the parser handles CREATE TABLE IDENTIFIER(...) and how SingleQuotedString values reach the IDENTIFIER function or tokenizer. Reproduce the example and determine whether the intended result is three quoted identifiers; done means the input is no longer treated as one literal and existing parsing behavior remains intact.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
compilers
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.