apache / apache/datafusion-sqlparser-rs

`Hive` dialect does not support `distribute by` or `sort by` in window specs

Open
#2,339 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
3.5k
Forks
772
Avg merge
4d 9h
Merged PRs (30d)
17

Description

In Hive 3.1.x, we found that the following query executes successfully:
```sql
select row_number() over(distribute by id sort by update_time desc) as row_num
from (select 1 as id, current_timestamp as update_time) t
```

Image

Image

However, when generating the AST using **the latest code from the main branch**, the following error occurs:

Image

Reference documentation:

https://github.com/apache/hive/blob/master/parser/src/java/org/apache/hadoop/hive/ql/parse/IdentifiersParser.g#L293-L297
https://github.com/apache/hive/blob/master/parser/src/java/org/apache/hadoop/hive/ql/parse/SelectClauseParser.g#L135-L139
https://github.com/apache/hive/blob/master/parser/src/java/org/apache/hadoop/hive/ql/parse/FromClauseParser.g#L271-L27

Contributor guide

No contributing guide indexed for this repository

Research direction

Reproduce the query in the Hive dialect and trace how its window specification is parsed. Compare the relevant grammar in Hive's IdentifiersParser.g, SelectClauseParser.g, and FromClauseParser.g, then add regression coverage showing that distribute by and sort by are accepted and the AST is generated successfully.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust, sql
Domain
compilers
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.