apache / apache/datafusion-sqlparser-rs

`is_alphabetic` or `is_ascii_alphabetic`?

Open
#1,418 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
3.5k
Forks
772
Avg merge
4d 9h
Merged PRs (30d)
17

Description

https://github.com/sqlparser-rs/sqlparser-rs/blob/4875dadbf5e691dd1e8ef179145829d8ca3ef4d8/src/dialect/mysql.rs#L32

In here, `is_alphabetic` checks for **Unicode Char with Alphabetic property**, which overlaps BMP(Basic Multilingual Plane) but not exclusive to BMP but also to astral plane and Han etc, should those be `is_ascii_alphabetic`?

other use of `is_alphabetic` have similiar problem, i.e. psql support a-z, but also letters with diacritical marks and non-Latin letters, which is kind of ambiguous though(presumably psql mean `is_alphabetic`?

so the current impl of mysql parser will accept chat not in BMP but is still Alphabetic(which is better than mysql standard I guess? but is inconsistent with mysql standard, since there are char in BMP that is not Alphabetic like `,`(FULLWIDTH COMMA)and Alphabetic Char not in BMP

refs to standards:
- unicode standard: https://www.unicode.org/Public/UCD/latest/ucd/DerivedCoreProperties.txt see `# Derived Property: Alphabetic` section
- mysql standard: https://dev.mysql.com/doc/refman/8.0/en/identifiers.html see
```
- Permitted characters in unquoted identifiers:

- ASCII: [0-9,a-z,A-Z$_] (basic Latin letters, digits 0-9, dollar, underscore)

- Extended: U+0080 .. U+FFFF
```
- psql standard: https://www.postgresql.org/docs/current/sql-syntax-lexical.html see `SQL identifiers and key words must begin with a letter (a-z, but also letters with diacritical marks and non-Latin letters) or an underscore (_)`

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with src/dialect/mysql.rs at line 32 and audit the other uses of `is_alphabetic` mentioned in the issue. Compare each dialect's identifier rules with the cited MySQL, PostgreSQL, and Unicode standards, then clarify the intended behavior and add regression coverage for the agreed character ranges.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust, sql
Domain
compilers, databases
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.