apache / apache/datafusion-sqlparser-rs

`PostgreSqlDialect` accepts large amounts of non-PostgreSQL syntax

Open
#2,237 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
3.5k
Forks
772
Avg merge
4d 9h
Merged PRs (30d)
17

Description

While building a parser correctness benchmark using libpg_query (`pg_query.rs`) as the PostgreSQL ground truth, we measured how often `PostgreSqlDialect` accepts SQL that real PostgreSQL rejects. The numbers are surprisingly high.

Against SQL extracted from the sqlparser-rs test suite itself:

- **28.7%** of statements rejected by pg_query are silently accepted by `PostgreSqlDialect` (37/129, PostgreSQL-specific test file)
- **30.0%** in the broader common-dialect test file (141/470)

We understand sqlparser-rs is intentionally permissive. The question is: **is this level of permissiveness intentional for `PostgreSqlDialect`, or is it leakage that would be worth tightening?**

## Examples of what `PostgreSqlDialect` currently accepts

A selection from the 141 cases found, grouped by the dialect the syntax originates from:

```sql
-- Oracle
FETCH NEXT IN my_cursor INTO result_table -- INTO clause on FETCH

-- SQL Server / T-SQL
SELECT TOP 3 * FROM tbl
EXEC my_proc N'param'
MERGE … OUTPUT inserted.* INTO log_target
EXECUTE FUNCTION f -- trigger EXECUTE without ()

-- MySQL / MariaDB
INSERT customer VALUES (1, 2, 3) -- missing INTO
INSERT OR REPLACE INTO t (id) VALUES(1)
DROP FUNCTION IF EXISTS f(a INTEGER, IN b INTEGER = 1) -- defaults in DROP

-- Snowflake / BigQuery
SELECT i FROM qt QUALIFY ROW_NUMBER() OVER (...) = 1
CREATE OR REPLACE TABLE t (a INT)
CREATE OR REPLACE USER IF NOT EXISTS u1 PASSWORD='secret'

-- ClickHouse
ALTER TABLE t ON CLUSTER my_cluster ADD CONSTRAINT bar PRIMARY KEY (baz)

-- HiveQL
ALTER TABLE t SET TBLPROPERTIES('classification' = 'parquet')

-- Unclear origin / possibly over-permissive parsing
ALTER TABLE t ALTER COLUMN id ADD GENERATED AS IDENTITY -- missing ALWAYS/BY DEFAULT
COPY t FROM 'f.csv' BINARY DELIMITER ',' CSV HEADER -- mutually exclusive formats
SHOW search_path search_path -- duplicate trailing token
```

Happy to help with PRs if the direction is clear.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with pg_query.rs as the PostgreSQL ground truth and the PostgreSQL-specific and common-dialect test files, then compare the listed accepted statements with their expected behavior. First establish whether PostgreSqlDialect should reject these cases and define a bounded scope; done means an agreed direction plus regression coverage for the selected syntax.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust, sql
Domain
compilers, databases
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.