alecthomas / alecthomas/participle

Proposal: Error tolerant parsing

Abierto
#102 4 comentarios 2 reacciones 0 asignados Ver en GitHub
proposal
Lenguaje dominante
Go
Estrellas
3.9k
Forks
213
Métricas de merge de PR
Sin PR fusionados en 30 d

Descripción

Participle currently supports returning a partial AST when parsing fails, but only up to the point of failure. This issue proposes that Participle reject failed tokens but continue to parse, if possible.

(transcribed from [Slack](https://gophers.slack.com/archives/CN9DS8YF3/p1598489311041100))

For participle to produce a full AST even when errored, my initial thought is that it would need to combinatorially match tokens to all branches of the parse tree, back to the root

eg. (hypothesising here) given this kind of input

```
a = "foo" bar
c = 10
```

and this grammar

```go
type Assignments struct {
Assignment []*Assignment `@@*`
}
type Assignment struct {
Ident string `@Ident "="`
Value *Value `@@`
}
type Value struct {
String *string ` @String`
Number *float64 `| @Number`
}
```

Once it hits the tokens `[bar, c, =, 10]`, `bar` would correctly parse into `Assignment.Ident` but `Value` would fail, so the parser would perhaps reset back to the start of Assignment, but reject the token `bar` as an error.

There'd have to be some mechanism for capturing the failed branches. Probably out of band somehow, or possibly the partially valid sub-tree could be inserted with a marker indicating it's an error. Similar to how Pos lexer.Position is special cased there could be something like `ParseFailure lexer.Position` that is populated if that branch has errors.

It would have to be opt-in though, as that kind of backtracking could be quite expensive.

I don't like that ParseFailure. What might be better is `RejectedTokens []lexer.Token`. It might not be ideal to require that be part of the AST 🤔.

So the AST in this case might end up something like:

```go
Assignments{
RejectedTokens []lexer.Token{...},
Assignment: []*Assignment{
Assignment{ /* a = "foo" */ },
Assignment{ /* bar */, RejectedTokens: ...},
Assignment{ /* c = 10 */ },
},
}
```

The RejectedTokens could be populated all the way back to the root, potentially.

Guía de contribución

No hay ninguna guía de contribución indexada para este repositorio

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.