apache / apache/datafusion-sqlparser-rs

Become more robust to parse errors

Open
#672 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
3.5k
Forks
772
Avg merge
4d 9h
Merged PRs (30d)
17

Description

I've started working on a sql [lsp server implementation](https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/), using `sqlparser` as the parser. Unfortunately, I very quickly hit a wall, mostly around how `sqlparser` discovers and surfaces errors. For instance, let's say the user types the following: `SELECT * F` and hits the key. A good lsp server would suggest completing `F` to `FROM`, even just do it automatically in that case. If one tries to parse that input string with `sqlparser`, it returns a `ParserError("Expected end of statement, found: F")`. The error isn't unexpected, but now (a) since the ast isn't available in the face of errors, there are no recommendations/commands/lints I can make based on a error-prone AST, and (b) I don't have enough information (line/col info) to send a diagnostic (the thing that makes the red squiggly lines) to the client. The line/col information was suggested in #179, but a fair amount of the work that would enable a really good LSP server implementation is being able to work with an AST that has errors in it.

I _suspect_ what I'm describing is _really_ hard, and has all sorts of backwards-incompatible changes that would be required to make this work. I think it'd be worthwhile, though.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading the sqlparser ParserError handling and the parser entry points involved when parsing `SELECT * F`; the issue provides no specific files or tests. Compare the current behavior with the LSP diagnostic requirements and determine how an error-tolerant AST, completion support, and line/column diagnostics should be defined. Done should include agreed behavior and coverage for incomplete SQL such as this example.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust, sql
Domain
compilers
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.