apache / apache/datafusion-sqlparser-rs
Become more robust to parse errors
- Dominant language
- Rust
- Stars
- 3.5k
- Forks
- 772
- Avg merge
- 4d 9h
- Merged PRs (30d)
- 17
Description
I've started working on a sql [lsp server implementation](https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/), using `sqlparser` as the parser. Unfortunately, I very quickly hit a wall, mostly around how `sqlparser` discovers and surfaces errors. For instance, let's say the user types the following: `SELECT * F` and hits the key. A good lsp server would suggest completing `F` to `FROM`, even just do it automatically in that case. If one tries to parse that input string with `sqlparser`, it returns a `ParserError("Expected end of statement, found: F")`. The error isn't unexpected, but now (a) since the ast isn't available in the face of errors, there are no recommendations/commands/lints I can make based on a error-prone AST, and (b) I don't have enough information (line/col info) to send a diagnostic (the thing that makes the red squiggly lines) to the client. The line/col information was suggested in #179, but a fair amount of the work that would enable a really good LSP server implementation is being able to work with an AST that has errors in it.
I _suspect_ what I'm describing is _really_ hard, and has all sorts of backwards-incompatible changes that would be required to make this work. I think it'd be worthwhile, though.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reading the sqlparser ParserError handling and the parser entry points involved when parsing `SELECT * F`; the issue provides no specific files or tests. Compare the current behavior with the LSP diagnostic requirements and determine how an error-tolerant AST, completion support, and line/column diagnostics should be defined. Done should include agreed behavior and coverage for incomplete SQL such as this example.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust, sql
- Domain
- compilers
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100