alecthomas / alecthomas/participle
Proposal: Error tolerant parsing
- Lingua principale
- Go
- Stelle
- 3.9k
- Fork
- 213
- Metriche di merge delle PR
- Nessuna PR unita negli ultimi 30g
Descrizione
Participle currently supports returning a partial AST when parsing fails, but only up to the point of failure. This issue proposes that Participle reject failed tokens but continue to parse, if possible.
(transcribed from [Slack](https://gophers.slack.com/archives/CN9DS8YF3/p1598489311041100))
For participle to produce a full AST even when errored, my initial thought is that it would need to combinatorially match tokens to all branches of the parse tree, back to the root
eg. (hypothesising here) given this kind of input
```
a = "foo" bar
c = 10
```
and this grammar
```go
type Assignments struct {
Assignment []*Assignment `@@*`
}
type Assignment struct {
Ident string `@Ident "="`
Value *Value `@@`
}
type Value struct {
String *string ` @String`
Number *float64 `| @Number`
}
```
Once it hits the tokens `[bar, c, =, 10]`, `bar` would correctly parse into `Assignment.Ident` but `Value` would fail, so the parser would perhaps reset back to the start of Assignment, but reject the token `bar` as an error.
There'd have to be some mechanism for capturing the failed branches. Probably out of band somehow, or possibly the partially valid sub-tree could be inserted with a marker indicating it's an error. Similar to how Pos lexer.Position is special cased there could be something like `ParseFailure lexer.Position` that is populated if that branch has errors.
It would have to be opt-in though, as that kind of backtracking could be quite expensive.
I don't like that ParseFailure. What might be better is `RejectedTokens []lexer.Token`. It might not be ideal to require that be part of the AST 🤔.
So the AST in this case might end up something like:
```go
Assignments{
RejectedTokens []lexer.Token{...},
Assignment: []*Assignment{
Assignment{ /* a = "foo" */ },
Assignment{ /* bar */, RejectedTokens: ...},
Assignment{ /* c = 10 */ },
},
}
```
The RejectedTokens could be populated all the way back to the root, potentially.
Guida per i contributori
Nessuna guida per i contributori indicizzata per questo repository
Valutazione
Questa issue non è ancora stata valutata.